The OsteoDetect software is a computer-aided detection and diagnostic software that uses an artificial intelligence algorithm to analyze two-dimensional X-ray images for signs of distal radius fracture, a common type of wrist fracture. The software marks the location of the fracture on the image to aid the provider in detection and diagnosis.

OsteoDetect analyzes wrist radiographs using machine learning techniques to identify and highlight regions of distal radius fracture during the review of posterior-anterior (front and back) and medial-lateral (sides) X-ray images of adult wrists. OsteoDetect is intended to be used by clinicians in various settings, including primary care, emergency medicine, urgent care and specialty care, such as orthopedics. It is an adjunct tool and is not intended to replace a clinician’s review of the radiograph or his or her clinical judgment.

“BriefCase uses an artificial intelligence algorithm to analyze images and highlight cases with detected ICH on a standalone desktop application in parallel to the ongoing standard of care image interpretation. The user is presented with notifications for cases with suspected ICH findings. Notifications include compressed preview images that are meant for informational purposes only and not intended for diagnostic use beyond notification. The device does not alter the original medical image and is not intended to be used as a diagnostic device. “

Our group evaluated the performance of a convolutional neural network (CNN) model developed by Aidoc (Tel Aviv, Israel). This model is one of the first artificial intelligence devices to receive FDA clearance for enabling radiologists to triage patients after scan acquisition. The algorithm was tested on 7112 non-contrast head CTs acquired during 2016–2017 from a two, large urban academic and trauma centers. Ground truth labels were assigned to the test data per PACS query and prior reports by expert neuroradiologists. No scans from these two hospitals had been used during the algorithm training process and Aidoc staff were at all times blinded to the ground truth labels. Model output was reviewed by three radiologists and manual error analysis performed on discordant findings. Specificity was 99%, sensitivity was 95%, and overall accuracy was 98%.”
The Utility of Deep Learning: Evaluation of a Convolutional Neural Network for Detection of Intracranial Bleeds on Non-Contrast Head Computed Tomography Studies 
Ojeda P, Zawaideh M et al.
Medical Imaging 2019: Image Processing, edited by Elsa D. Angelini, Bennett A. Landman, Proc. of SPIE Vol. 10949, 109493J

“Specificity was 99%, sensitivity was 95%, and overall accuracy was 98%.”
The Utility of Deep Learning: Evaluation of a Convolutional Neural Network for Detection of Intracranial Bleeds on Non-Contrast Head Computed Tomography Studies 
Ojeda P, Zawaideh M et al.
Medical Imaging 2019: Image Processing, edited by Elsa D. Angelini, Bennett A. Landman, Proc. of SPIE Vol. 10949, 109493J

Conclusions: TAT was reduced in the month following AI implementation among all categories of head CT. Overall, there was a 24.5% reduction in TAT and a slightly greater reduction of 37.8% in all ICH-positive cases, suggesting the positive cases were prioritized. This effect extended to ED and inpatient studies. The reduction in overall TAT may reflect the disproportionate influence positive cases had on overall TAT, or that the AI widget also flagged some cases which were not positive, resulting in the prioritization of all NCCT over other exams. 
Comparison of After-Hours Head CT Report Turnaround Time Before and After Implementation of an Artificial Intelligence Widget Developed to Detect Intracranial Hemorrhage. 
Brady Laughlin et al.
American College of Radiology(2018)

courtesy of AIDoc

“We trained a fully convolutional neural network with 4,396 head CT scans performed at the University of California at San Francisco and affiliated hospitals and compared the algorithm’s performance to that of 4 American Board of Radiology (ABR) certified radiologists on an independent test set of 200 randomly selected head CT scans. Our algorithm demonstrated the highest accuracy to date for this clinical application, with a receiver operating characteristic (ROC) area under the curve (AUC) of 0.991 ± 0.006 for identification of examinations positive for acute intracranial hemorrhage, and also exceeded the performance of 2 of 4 radiologists.” 
Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning 
Kuo W et al.
Proc Natl Acad Sci U S A. 2019 Nov 5;116(45):22737-22745

A prioritized “STAT” X-ray can sit waiting for up to eight hours for a radiologist’s review1. However, when a patient is scanned on a device with Critical Care Suite, the system automatically analyzes the images by simultaneously searching for a pneumothorax. If a pneumothorax is suspected, an alert – along with the original chest X-ray – is sent directly to the radiologist for review via picture archiving and communication systems (PACS). The technologist also receives a subsequent on-device notification to give awareness of the prioritized cases. Quality-focused AI algorithms simultaneously analyze and flag protocol and field of view errors as well as auto rotate the images on-device. 

”The FDA has recently approved software by AIDoc Medical (Tel Aviv, Israel) as well as Zebra Medical Vision (Shefayim, Israel) that automatically detects pulmonary embolisms in chest CTs. As described by Weikert et al., the work by AIDoc was based on a compiled dataset of 1499 CT pulmonary angiograms with corresponding reports that was then tested on four trained prototype algorithms. The algorithm that achieved optimal results was shown to have a sensitivity of 93% and a specificity of 95%, with a positive predictive value of 77%.”
The first use of artificial intelligence (AI) in the ER: triage not diagnosis 
Edmund M. Weisberg, Linda C. Chu, Elliot K. Fishman
Emergency Radiology (2020) 27:361–366

”The ability to triage patients and take care of acute processes such as intracranial bleed, pneumothorax, and pulmonary embolism will largely benefit the health system, improving patient care and reducing costs. In the end, our mission is the care of our patients, and if AI can improve it, we will need to adopt it with open arms.”
The first use of artificial intelligence (AI) in the ER: triage not diagnosis 
Edmund M. Weisberg, Linda C. Chu, Elliot K. Fishman
Emergency Radiology (2020) 27:361–366

Background: The interpretation of radiographs suffers from an ever-increasing workload in emergency and radiology departments, while missed fractures represent up to 80% of diagnostic errors in the emergency department. 
Purpose: To assess the performance of an artificial intelligence (AI) system designed to aid radiologists and emergency physicians in the detection and localization of appendicular skeletal fractures.
Conclusion: The artificial intelligence aid provided a gain of sensitivity (8.7% increase) and specificity (4.1% increase) without loss of reading speed. 
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study 
Loïc Duron et al. 
Radiology 2021; 300:120–129 

Materials and Methods: The AI system was previously trained on 60 170 radiographs obtained in patients with trauma. The radio- graphs were randomly split into 70% training, 10% validation, and 20% test sets. Between 2016 and 2018, 600 adult patients in whom multiview radiographs had been obtained after a recent trauma, with or without one or more fractures of shoulder, arm, hand, pelvis, leg, and foot, were retrospectively included from 17 French medical centers. Radiographs with quality precluding hu- man interpretation or containing only obvious fractures were excluded. Six radiologists and six emergency physicians were asked to detect and localize fractures with (n = 300) and fractures without (n = 300) the aid of software highlighting boxes around AI- detected fractures. Aided and unaided sensitivity, specificity, and reading times were compared by means of paired Student t tests after averaging of performances of each reader. 
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study 
Loïc Duron et al. 
Radiology 2021; 300:120–129  

Results: A total of 600 patients (mean age 6 standard deviation, 57 years 6 22; 358 women) were included. The AI aid improved the sensitivity of physicians by 8.7% (95% CI: 3.1, 14.2; P = .003 for superiority) and the specificity by 4.1% (95% CI: 0.5, 7.7; P < .001 for noninferiority) and reduced the average number of false-positive fractures per patient by 41.9% (95% CI: 12.8, 61.3; P = .02) in patients without fractures and the mean reading time by 15.0% (95% CI: 230.4, 3.8; P = .12). Finally, stand-alone perfor- mance of a newer release of the AI system was greater than that of all unaided readers, including skeletal expert radiologists, with an area under the receiver operating characteristic curve of 0.94 (95% CI: 0.92, 0.96). 
Conclusion: The artificial intelligence aid provided a gain of sensitivity (8.7% increase) and specificity (4.1% increase) without loss of reading speed. 
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study 
Loïc Duron et al. 
Radiology 2021; 300:120–129 

“In conclusion, we showed that a deep learning algorithm aided emergency physicians and radiologists in improving their diagnostic performance and boosting their time efficiency in the localization of all appendicular bone fractures on plain radiographs. The algorithm improved as updates were made, which bodes well for helping physicians cope with the increasing work- load more effectively, and an evaluation in future prospective studies will be needed.”
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study 
Loïc Duron et al. 
Radiology 2021; 300:120–129 

”Our study had several limitations. First, readers and the AI sys- tem were assessed on their ability to make decisions based on image analysis alone, without knowledge about the findings from the patients’ physical examination or their medical history, creating a context bias. Clinical data can be crucial in making decisions; however, in our experience radiologists often lack relevant clinical data. Second, a Hawthorne effect may have affected the performances of readers, that is, a modification of their behavior in response to their awareness of being observed for the research project, leading, for instance, to a more thorough reading than in clinical practice. Similarly, cognitive biases related to the emer- gency setting could not be replicated in a retrospective study.”
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study 
Loïc Duron et al. 
Radiology 2021; 300:120–129 

OBJECTIVES To develop a deep learning–based algorithm that can classify normal and abnormal results from chest radiographs with major thoracic diseases including pulmonary malignant neoplasm, active tuberculosis, pneumonia, and pneumothorax and to validate the algorithm’s performance using independent data sets. 
CONCLUSIONS AND RELEVANCE The algorithm consistently outperformed physicians, including thoracic radiologists, in the discrimination of chest radiographs with major thoracic diseases, demonstrating its potential to improve the quality and efficiency of clinical practice. 
Development and Validation of a Deep Learning–Based Automated Detection Algorithm for Major Thoracic Diseases on Chest Radiographs 
Eui Jin Hwang et al. 
JAMA Network Open. 2019;2(3):e191095. doi:10.1001/jamanetworkopen.2019.1095

Findings In this diagnostic study of54 221 chest radiographs with normal findings and 35 613 with abnormal findings, the deep learning–based algorithm for discrimination of chest radiographs with pulmonary malignant neoplasms, active tuberculosis, pneumonia, or pneumothorax demonstrated excellent and consistent performance throughout 5 independent data sets. The algorithm outperformed physicians, including radiologists, and enhanced physician performance when used as a second reader. 
Development and Validation of a Deep Learning–Based Automated Detection Algorithm for Major Thoracic Diseases on Chest Radiographs 
Eui Jin Hwang et al. 
JAMA Network Open. 2019;2(3):e191095. doi:10.1001/jamanetworkopen.2019.1095

“The high performance of the DLAD in classification of CRs with normal and abnormal findings indicative of major thoracic diseases, outperforming even thoracic radiologists, suggests its potential for stand-alone use in select clinical situations. It may also help improve the clinical workflow by prioritizing CRs with suspicious abnormal findings requiring prompt diagnosis and management. It can also improve radiologists’ work efficiency, which would partially alleviate the heavy workload burden that radiologists face today and improve patients’ turnaround time.”
Development and Validation of a Deep Learning–Based Automated Detection Algorithm for Major Thoracic Diseases on Chest Radiographs 
Eui Jin Hwang et al. 
JAMA Network Open. 2019;2(3):e191095. doi:10.1001/jamanetworkopen.2019.1095

Background: Radiology reporting of emergency whole-body computed tomography (CT) scans is time- critical and therefore involves a significant risk of pathology under-detection. We hypothesize a relevant number of initially missed secondary thoracic findings that would have been detected by an artificial intelligence (AI) software platform including several pathology-specific AI algorithms. 
Methods: This retrospective proof-of-concept-study consecutively included 105 shock-room whole-body CT scans. Image data was analyzed by platform-bundled AI-algorithms, findings were reviewed by radiology experts and compared with the original radiologist’s reports. We focused on secondary thoracic findings, such as cardiomegaly, coronary artery plaques, lung lesions, aortic aneurysms and vertebral fractures. 
Reduction of missed thoracic findings in emergency whole-body computed tomography using artificial intelligence assistance 
Johannes Rueckel et al.
Quant Imaging Med Surg 2021;11(6):2486-2498

“Based on 105 “shock-room” emergency CT scans, we demonstrated an AI system that would have decreased the number of missed secondary thoracic findings in an AI- assisted reading setting. The added clinical value could be quantified by the number of additional findings as follows: up to 25 (23.8%) patients with cardiomegaly or borderline heart size, 17 (16.2%) patients with coronary plaques, 34 (32.4%) patients with dilatations of the thoracic aorta, 13 additional vertebral fractures (two of them with an acute traumatic origin) and three lung lesions of two different patients that were radiologically classified as “in recommendation to control”.”
Reduction of missed thoracic findings in emergency whole-body computed tomography using artificial intelligence assistance 
Johannes Rueckel et al.
Quant Imaging Med Surg 2021;11(6):2486-2498

“In conclusion, we demonstrated in a retrospective proof- of-concept setting the high potential of AI approaches to reduce the number of missed secondary findings in clinical emergency settings that require a very time-critical radiological reporting. In particular, the integration of different specialized algorithms in a single software solution is promising to avoid clinically too narrow AI applications. But also with regard to less urgent applications of medical imaging, it should be mentioned that especially non- radiology clinicians might even take more benefit from AI- assisted image analysis compared to anyway well-trained radiologists, e.g., in clinical settings without 24/7 radiology coverage or long turnaround times for radiology reporting.”
Reduction of missed thoracic findings in emergency whole-body computed tomography using artificial intelligence assistance 
Johannes Rueckel et al.
Quant Imaging Med Surg 2021;11(6):2486-2498

”Once a patient has left the hospital, it is incumbent on a health care system to ensure appropriate follow-up care. Between subspecialty follow-up appointments and imaging follow-up recommendations, closing the loop on an emergency department visit is an area of active development with modern advanced computing systems. There are existing plat- forms which utilize Natural language processing (NLP) and AI/ML to analyze radiology and pathology reports in real time to ensure existing best practice guidelines are being utilized and provided to ordering physicians. These systems not only guide the report generation process but also have limited ability to monitor outpatient compliance and provide alerts/ reminders to patients and primary care physicians to encourage adherence to follow-up guidelines.”
Applications of artificial intelligence in the emergency department. 
Moulik SK, Kotter N, Fishman EK.
Emerg Radiol. 2020 Aug;27(4):355-358

”AI/ML systems function through the use of mathematical models which can be trained on var- ious datasets. Systems are being established for allowing general AI/ML models to be modified to better reflect localized communities through the use of techniques known as transfer learning and federated learning. In the future, AI/ML systems will likely get continuous or frequent updates of their global models while being able to maintain the specialized training that allows a general model to be broadly applicable within a narrow subgroup or community.”
Applications of artificial intelligence in the emergency department. 
Moulik SK, Kotter N, Fishman EK.
Emerg Radiol. 2020 Aug;27(4):355-358

“I observe that the expectations from AI and radiologists are fundamentally different. The expectations of AI are based on a strong and justified mistrust about the way that AI makes decisions. Because AI decisions are not well understood, it is difficult to know how the algorithms will behave in new, unexpected situations. However, this mistrust is not mirrored in our expectations of human readers. Despite well-proven idiosyncrasies and biases in human decision- making, we take comfort from the assumption that others make decisions in a way as we do, and we trust our own decision-making. Despite poor ability to explain decision- making processes in humans, we accept explanations of decisions given by other humans.” 
Do We Expect More from Radiology AI than from Radiologists? 
Mazurowski, MA
Radiology: Artificial Intelligence 2021; 3(4):e200221

On the one hand, we should, to a reasonable extent, mistrust AI to make correct decisions regardless of the setting. Therefore, it is crucial that we test the algorithms in real-world scenarios and on datasets that are diverse in terms of patient populations, scanner parameters, imaging protocols, technologists who ac- quire the images, and any other parameters that may reasonably affect the decision.
On the other hand, putting an undue burden on the algorithms because we trust ourselves as human beings may not be a correct policy. The goal should be to implement a radiologic practice that provides the most benefit to the patients. This needs to be based on a reasonable mistrust toward the algorithm, but it should not be based on an unjustified trust toward our own human ability to perceive and make decisions. that we place in our fellow human beings.”
Do We Expect More from Radiology AI than from Radiologists? 
Mazurowski, MA
Radiology: Artificial Intelligence 2021; 3(4):e200221

“EHRs were quickly installed without strong evidence to guide their design, implementation, and regulation, and have contributed to a highly transactional model, with care signified by tick boxes, communication by smart phrases, and where screen-time has replaced face-time as he primary act of healthcare. It is no wonder widespread burnout among physicians has resulted.”
Ten Ways Artificial Intelligence Will Transform Primary Care 
Steven Y. Lin, Megan R. Mahoney, Christine A. Sinsky 
J Gen Intern Med 34(8):1626–30

“Undivided attention with compassion is the most powerful diagnostic and therapeutic tool physicians can provide their patients. AI will be most effective when it enhances physicians’ ability to focus their full attention on the patient by shifting the physicians’ responsibilities away from transactional tasks toward personalized care that lies at the heart of human healing.”
Ten Ways Artificial Intelligence Will Transform Primary Care 
Steven Y. Lin, Megan R. Mahoney, Christine A. Sinsky 
J Gen Intern Med 34(8):1626–30

“In conclusion, there is little doubt that AI technology will benefit almost all medical personnel, ranging from specialty physicians to paramedics, in the future. Furthermore, patients should benefit from AI technology directly via mobile applications. Physicians should collaborate with the different stakeholders within the AI ecosystem to provide ethical, practical, user-friendly, and cost-effective solutions that reduce the gap between research settings and applications in clinical practice. Collaborations with regulators, patient advocates, AI companies, technology giants, and venture capitalists will help move the field forward.”
A New Dawn for the Use of Artificial Intelligence in Gastroenterology, Hepatology and Pancreatology
Akihiko Oka , Norihisa Ishimura and Shunji Ishihara 
Diagnostics 2021, 11, 1719. https://doi.org/10.3390/diagnostics11091719