Imaging Pearls ❯ Trauma ❯ Deep Learning and AI
|
-- OR -- |
|
- Purpose There is a growing body of diagnostic performance studies for emergency radiology-related artificial intelligence/ machine learning (AI/ML) tools; however, little is known about user preferences, concerns, experiences, expectations, and the degree of penetration of AI tools in emergency radiology. Our aim is to conduct a survey of the current trends, perceptions, and expectations regarding AI among American Society of Emergency Radiology (ASER) members.
Conclusion ASER member respondents are in general optimistic about the impact of AI in the practice of emergency radiology and its impact on the popularity of emergency radiology as a subspecialty. The majority expect to see transparent and explainable AI models with the radiologist as the decision-maker.
A survey of ASER members on artificial intelligence in emergency radiology: trends, perceptions, and expectations
Anjali Agrawal et al.
Emergency Radiology (2023) 30:267–277 - “Just over half of respondents among the ASER membership currently use commercial AI tools in their practice. Two thirds of respondents who currently use AI tools feel that they improve quality of care, and most find themselves disagreeing with AI predictions in 5–20% of studies. Concerns and apprehensions pertaining to overdiagnosis and generalization to their local patient populations are shared by over half of end-users. The majority of respondents expect to see transparent and explainable AI tools with the onus of the final decision with the radiologist.”
A survey of ASER members on artificial intelligence in emergency radiology: trends, perceptions, and expectations
Anjali Agrawal et al.
Emergency Radiology (2023) 30:267–277 - Purpose To map the evolution and current state of trauma radiology CAD tools along key dimensions of technology readiness.
Conclusions Trauma imaging CAD tools are likely to improve patient care but are currently in an early stage of maturity, with few FDA-approved products for a limited number of uses. The scarcity of high-quality annotated data remains a major barrier.
Artificial intelligence CAD tools in trauma imaging: a scoping review from the American Society of Emergency Radiology (ASER) AI/ML Expert Panel
David Dreizin et al.
Emergency Radiology (2023) 30:251–265 - “Approximately 84% of studies described siloed datasets with fewer than 5000 patients. Cross-sectional imaging datasets for abdominopelvic and chest trauma ranged from fewer than 100 to 778 patients, and no commercial products were described in these domains. Torso pathology including organ injury, contusion, and hemorrhage is highly variable in size and appearance with small target to volume ratios, and multiscale DL-based tools for torso pathology have been late-comers and were not reported for trauma until 2020.”
Artificial intelligence CAD tools in trauma imaging: a scoping review from the American Society of Emergency Radiology (ASER) AI/ML Expert Panel
David Dreizin et al.
Emergency Radiology (2023) 30:251–265 - “Even though trauma remains the leading cause of death and disability in patients under 45 years of age, trauma imaging remains a relatively small and underfunded branch of radiology. In the field of radiology as a whole, AI/ML publications have increased exponentially, primarily in the fields of neuroradiology, abdominal imaging, and chest imaging, spurred by federal agency and industry-side funding. Our findings suggest that increased funding opportunities, researcher engagement, research training, and institutional buy-in will accelerate research productivity and translation of tools to the trauma setting.”
Artificial intelligence CAD tools in trauma imaging: a scoping review from the American Society of Emergency Radiology (ASER) AI/ML Expert Panel
David Dreizin et al.
Emergency Radiology (2023) 30:251–265 - ”In conclusion, AI CAD tools are likely to improve ER/ trauma radiologist productivity and diagnostic performance, reduce turnaround times, decrease ER and hospital stays, and improve survival of severely injured patients. However, these tools are currently in a very early stage of maturity. There are few FDA-approved products for a limited number of use cases, and there has not been sufficient validation of commercial tools to generate meta-analyses. The scarcity of large heterogeneous datasets with high-quality annotation continues to pose a major barrier. There remains an unmet need for out-of-the-box tools that accelerate data labeling and for multicenter privacy-preserving distributed learning.”
Artificial intelligence CAD tools in trauma imaging: a scoping review from the American Society of Emergency Radiology (ASER) AI/ML Expert Panel
David Dreizin et al.
Emergency Radiology (2023) 30:251–265 - “A greater emphasis should be placed on performance validation data that incorporates assessment of bias and robustness across relevant subgroups. The methodology used for ground truth annotation is highly variable across the body of literature in this area. Researchers should be encouraged to employ independent readers with arbitration and provide data on reader agreement and reproducibility of measurements. Additionally, the range of techniques for explainability and interpretability using scalable DL-based approaches remains narrow, and methods that build trust through human–computer interaction are lacking. More emphasis should be placed on evaluation of end-user assessment of system benevolence and capability. Finally, an increase in funding opportunities would likely accelerate the R&D pipeline for trauma imaging CAD tools.”
Artificial intelligence CAD tools in trauma imaging: a scoping review from the American Society of Emergency Radiology (ASER) AI/ML Expert Panel
David Dreizin et al.
Emergency Radiology (2023) 30:251–265
- AI is perhaps ideal for the ER where the diagnostic process is compressed in time and often location
- order entry for imaging studies optimized by AI (Step # 1)
- image protocolling streamlined by AI (Step # 2)
- image acquisition supported by AI (Step # 3)
- image post-processing supported by AI (Step # 4)
- decision support provided by AI (Step # 5)
- clinical decision support provided by AI integration (Step # 6)
Exploring the Role of Artificial Intelligence in an Emergency and Trauma Radiology Department
Sabeena Jalal et al.
Canadian Association of Radiologists’ Journal 2021, Vol. 72(1) 167-174 - Step 1: order entry for imaging studies optimized by AI
- Protocols designed for clinical presentation (i.e. stroke, trauma)
- Protocols designed based on evaluation in the ambulance (i.e. acute abdomen, possible dissection
- Optimal use of imaging resources (and preventing overuse) - Step 2: image protocolling streamlined by AI
- Selecting the right study (CT vs MR vs US) as well as designing the right protocol is critical
- Errors in designing ER protocols for CT can result in suboptimal studies (i.e. wrong phase/phases of acquisition - Step 3: image acquisition supported by AI
- Optimal scan protocols can be done with decreased interaction on a case by case basis with the radiologist and radiologic technologist - Step 4: image post-processing supported by AI
- add 3D especially cinematic rendering for the radiologist and referring clinician and the image generation is done with AI
- Additional post processing including volume calculations can be done with AI support - Step 5: decision support provided by AI
- Triage and work list optimization
- AI assisted detection of pneumothorax or pneumoperitoneum and spinal fractures
- AI detection of incidental findings and management of these findings - Step 6: clinical decision support provided by AI integration
- AI assisted clinical diagnosis and management decisions including recent work on sepsis
- The rle of wearable devices in the ER may help with monitoring patients especially when they go to Radiology for exams. - Challenges to AI in the ER
- Defining the problem in finite demensions to allow for product development
- Funding and resources in the current environment
- Availability of computing resources needed (its all about the people)
- Multidisciplinary challenges and turf battle issues - Background: The interpretation of radiographs suffers from an ever-increasing workload in emergency and radiology departments, while missed fractures represent up to 80% of diagnostic errors in the emergency department.
Purpose: To assess the performance of an artificial intelligence (AI) system designed to aid radiologists and emergency physicians in the detection and localization of appendicular skeletal fractures.
Conclusion: The artificial intelligence aid provided a gain of sensitivity (8.7% increase) and specificity (4.1% increase) without loss of reading speed.
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study
Loïc Duron et al.
Radiology 2021; 300:120–129 - Background: The interpretation of radiographs suffers from an ever-increasing workload in emergency and radiology departments, while missed fractures represent up to 80% of diagnostic errors in the emergency department.
Purpose: To assess the performance of an artificial intelligence (AI) system designed to aid radiologists and emergency physicians in the detection and localization of appendicular skeletal fractures.
Conclusion: The artificial intelligence aid provided a gain of sensitivity (8.7% increase) and specificity (4.1% increase) without loss of reading speed.
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study
Loïc Duron et al.
Radiology 2021; 300:120–129 - Materials and Methods: The AI system was previously trained on 60 170 radiographs obtained in patients with trauma. The radio- graphs were randomly split into 70% training, 10% validation, and 20% test sets. Between 2016 and 2018, 600 adult patients in whom multiview radiographs had been obtained after a recent trauma, with or without one or more fractures of shoulder, arm, hand, pelvis, leg, and foot, were retrospectively included from 17 French medical centers. Radiographs with quality precluding hu- man interpretation or containing only obvious fractures were excluded. Six radiologists and six emergency physicians were asked to detect and localize fractures with (n = 300) and fractures without (n = 300) the aid of software highlighting boxes around AI- detected fractures. Aided and unaided sensitivity, specificity, and reading times were compared by means of paired Student t tests after averaging of performances of each reader.
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study
Loïc Duron et al.
Radiology 2021; 300:120–129 - Results: A total of 600 patients (mean age 6 standard deviation, 57 years 6 22; 358 women) were included. The AI aid improved the sensitivity of physicians by 8.7% (95% CI: 3.1, 14.2; P = .003 for superiority) and the specificity by 4.1% (95% CI: 0.5, 7.7; P < .001 for noninferiority) and reduced the average number of false-positive fractures per patient by 41.9% (95% CI: 12.8, 61.3; P = .02) in patients without fractures and the mean reading time by 15.0% (95% CI: 230.4, 3.8; P = .12). Finally, stand-alone perfor- mance of a newer release of the AI system was greater than that of all unaided readers, including skeletal expert radiologists, with an area under the receiver operating characteristic curve of 0.94 (95% CI: 0.92, 0.96).
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study
Loïc Duron et al.
Radiology 2021; 300:120–129 - Results: A total of 600 patients (mean age 6 standard deviation, 57 years 6 22; 358 women) were included. The AI aid improved the sensitivity of physicians by 8.7% (95% CI: 3.1, 14.2; P = .003 for superiority) and the specificity by 4.1% (95% CI: 0.5, 7.7; P < .001 for noninferiority) and reduced the average number of false-positive fractures per patient by 41.9% (95% CI: 12.8, 61.3; P = .02) in patients without fractures and the mean reading time by 15.0% (95% CI: 230.4, 3.8; P = .12). Finally, stand-alone perfor- mance of a newer release of the AI system was greater than that of all unaided readers, including skeletal expert radiologists, with an area under the receiver operating characteristic curve of 0.94 (95% CI: 0.92, 0.96).
Conclusion: The artificial intelligence aid provided a gain of sensitivity (8.7% increase) and specificity (4.1% increase) without loss of reading speed.
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study
Loïc Duron et al.
Radiology 2021; 300:120–129 - Summary
The artificial intelligence aid improved the sensitivity and specificity of radiologists and emergency physicians in the localization of appendicular fractures on radiographs, with no additional reading time.
Key Results
• The artificial intelligence (AI) aid, which highlighted potential fractures on full-resolution radiographs, improved the sensitivity (8.7% increase, P = .006) and specificity (4.1% increase, P = .03) of emergency doctors and radiologists in the diagnosis of appen- dicular fractures.
• The stand-alone area under the receiver operating characteristic curve, requiring that the AI system detect the precise locations of all fractures on an examination, was .94 with a newer release of the AI system.
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study
Loïc Duron et al.
Radiology 2021; 300:120–129 
Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study
Loïc Duron et al.
Radiology 2021; 300:120–129- "Once a patient has left the hospital, it is incumbent on a health care system to ensure appropriate follow-up care. Between subspecialty follow-up appointments and imaging follow-up recommendations, closing the loop on an emergency department visit is an area of active development with modern advanced computing systems. There are existing plat- forms which utilize Natural language processing (NLP) and AI/ML to analyze radiology and pathology reports in real time to ensure existing best practice guidelines are being utilized and provided to ordering physicians. These systems not only guide the report generation process but also have limited ability to monitor outpatient compliance and provide alerts/ reminders to patients and primary care physicians to encourage adherence to follow-up guidelines.”
Applications of artificial intelligence in the emergency department.
Moulik SK, Kotter N, Fishman EK.
Emerg Radiol. 2020 Aug;27(4):355-358 - “Patients who presents to the ER needs to be rapidly triaged to appropriate acuity level, and when possible, the relevant diagnostic tests need to be ordered. In emergency departments the EMTALA screening is often performed by a midlevel practitioner who will assess the patient and order initial imaging and laboratory tests. This process is labor in- tensive and inherently limited in the ability of a practitioner to ingest and process a patient’s history, imaging, and prior labs in order to put the current visit in context. A rudimentary form of automation is used in these scenarios in the form of stan- dard order sets that the midlevel practitioner can select based on presenting complaint and assessment. Though expedient and relatively easy to implement, this process over-utilizes imaging and laboratory resources by creating overly broad categorizations of patients.”
Applications of artificial intelligence in the emergency department.
Moulik SK, Kotter N, Fishman EK.
Emerg Radiol. 2020 Aug;27(4):355-358 - "AI/ML systems function through the use of mathematical models which can be trained on var- ious datasets. Systems are being established for allowing general AI/ML models to be modified to better reflect localized communities through the use of techniques known as transfer learning and federated learning. In the future, AI/ML systems will likely get continuous or frequent updates of their global models while being able to maintain the specialized training that allows a general model to be broadly applicable within a narrow subgroup or community.”
Applications of artificial intelligence in the emergency department.
Moulik SK, Kotter N, Fishman EK.
Emerg Radiol. 2020 Aug;27(4):355-358
- “This study showed that a deep learning model can be trained to detect wrist fractures in radiographs with diagnostic accuracy similar to that of senior subspecialized orthopedic surgeons. Additionally, this study showed that, when emergency medicine clinicians are provided with the assistance of the trained model, their ability to detect wrist fractures can be significantly improved, thus diminishing diagnostic errors and also improving the clinicians’ efficiency."
Deep neural network improves fracture detection by clinicians
Lindsey R et al.
PNAS | November 6, 2018 | vol. 115 | no. 45 | 11591–11596 - “The approach of this investigation is to apply machine learning algorithms trained by experts in the field to less experienced clinicians (who are at particular risk for diagnostic errors yet responsible for primary patient care and triage) to improve both their performance and efficiency. The learning model presented in this study mitigates these factors.”
Deep neural network improves fracture detection by clinicians
Lindsey R et al.
PNAS | November 6, 2018 | vol. 115 | no. 45 | 11591–11596 - “This study shows that deep learning models offer potential for subspecialized clinicians (without machine learning experience) to teach computers how to emulate their diagnostic expertise and thereby help patients on a global scale. Although teaching the model is a laborious process requiring collecting thousands of radiographs and carefully labeling them, making a prediction using the trained model takes less than a second on a modern computer.”
Deep neural network improves fracture detection by clinicians
Lindsey R et al.
PNAS | November 6, 2018 | vol. 115 | no. 45 | 11591–11596 - “ Historically, computer-assisted detection (CAD) in radiology has failed to achieve improvements in diagnostic accuracy, decreasing clinician sensitivity and leading to unnecessary further diagnostic tests. With the advent of deep learning approaches to CAD, there is great excitement about its appli- cation to medicine, yet there is little evidence demonstrating improved diagnostic accuracy in clinically-relevant applica- tions. We trained a deep learning model to detect fractures on radiographs with a diagnostic accuracy similar to that of senior subspecialized orthopedic surgeons. We demonstrate that when emergency medicine clinicians are provided with the assistance of the trained model, their ability to accurately detect fractures significantly improves.”
Deep neural network improves fracture detection by clinicians
Robert Lindsey et al.
Proc Natl Acad Sci U S A. 2018 Nov 6;115(45):11591-11596 - In this work, we developed a deep neural network to detect and localize fractures in radiographs. We trained it to accurately emulate the expertise of 18 senior sub- specialized orthopedic surgeons by having them annotate 135,409 radiographs. We then ran a controlled experiment with emergency medicine clinicians to evaluate their ability to detect fractures in wrist radiographs with and without the assistance of the deep learning model. The average clinician’s sensitivity was 80.8% (95% CI, 76.7–84.1%) unaided and 91.5% (95% CI, 89.3–92.9%) aided, and specificity was 87.5% (95 CI, 85.3–89.5%) unaided and 93.9% (95% CI, 92.9–94.9%) aided. The average clinician experienced a relative reduction in misinterpretation rate of 47.0% (95% CI, 37.4– 53.9%).
Deep neural network improves fracture detection by clinicians
Robert Lindsey et al.
Proc Natl Acad Sci U S A. 2018 Nov 6;115(45):11591-11596 - “The significant improvements in diagnostic accuracy that we observed in this study show that deep learning methods are a mechanism by which senior medical specialists can deliver their expertise to generalists on the front lines of medicine, thereby providing substantial improvements to patient care.”
Deep neural network improves fracture detection by clinicians
Robert Lindsey et al.
Proc Natl Acad Sci U S A. 2018 Nov 6;115(45):11591-11596 - Misinterpretation of radiographs may have grave consequences, resulting in complications including malunion with restricted range of motion, posttraumatic osteoarthritis, and joint collapse, the latter of which may require joint replacement. Misdiagnoses are also the primary cause of malpractice claims or litigation. There are multiple factors that can contribute to radiographic misinterpretations of fractures by clinicians, including physician fatigue, lack of subspecialized expertise, and inconsistency among reading physicians.
Deep neural network improves fracture detection by clinicians
Robert Lindsey et al.
Proc Natl Acad Sci U S A. 2018 Nov 6;115(45):11591-11596 - “The approach of this investigation is to apply machine learning algorithms trained by experts in the field to less experienced clinicians (who are at particular risk for diagnostic errors yet responsible for primary patient care and triage) to improve both their performance and efficiency. The learning model presented in this study mitigates these factors. It does not become fatigued, it always provides a consistent read, and it gains subspecialized expertise by being provided with labeled radiographs from human experts.”
Deep neural network improves fracture detection by clinicians
Robert Lindsey et al.
Proc Natl Acad Sci U S A. 2018 Nov 6;115(45):11591-11596 - Thus, we speculate that, someday, technology may permit any patient whose clinician has computer access to receive the same high-quality radiographic interpretations as those received by the patients of senior subspecialized experts.
Deep neural network improves fracture detection by clinicians
Robert Lindsey et al.
Proc Natl Acad Sci U S A. 2018 Nov 6;115(45):11591-11596