google ads
Search

Everything you need to know about Computed Tomography (CT) & CT Scanning

Deep Learning: Radiomics Imaging Pearls - Educational Tools | CT Scanning | CT Imaging | CT Scan Protocols - CTisus
Imaging Pearls ❯ Deep Learning ❯ Radiomics

-- OR --

  • Purpose: The purpose of this study is to evaluate diagnostic performance of a commercially available radiomics research prototype vs. an in-house radiomics software in the binary classification of CT images from patients with pancreatic ductal adenocarcinoma (PDAC) vs. healthy controls.
    Results: When 40 radiomics features were used in the random forest classification, in-house software achieved superior sensitivity (1.00) and accuracy (0.992) compared to the commercially available research prototype (sensitivity = 0.950, accuracy = 0.968). When the number of features was reduced to five features, diagnostic performance of the in-house soft- ware decreased to sensitivity (0.950), specificity (0.923), and accuracy (0.936). Diagnostic performance of the commercially available research prototype was unchanged.
    Conclusion: Commercially available and in-house radiomics software achieve similar diagnostic performance, which may lower the barrier of entry for radiomics research and allow more clinician-scientists to perform radiomics research.
    Diagnostic performance of commercially available vs. in‐house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls
    Linda C. Chu · Berkan Solmaz · Seyoun Park · Satomi Kawamoto · Alan L. Yuille · Ralph H. Hruban · Elliot K. Fishman
    Abdominal Radiology (2020) 45:2469–2475
  • “This study showed that a commercially available radiomics software may be able to achieve similar diagnostic performance as an in-house radiomics software. The results obtained from one radiomics software may be transferrable to another system. Availability of commercial radiomics software may lower the barrier of entry for radiomics research and allow more researchers to engage in this exciting area of research.”
    Diagnostic performance of commercially available vs. in‐house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls
    Linda C. Chu · Berkan Solmaz · Seyoun Park · Satomi Kawamoto · Alan L. Yuille · Ralph H. Hruban · Elliot K. Fishman
    Abdominal Radiology (2020) 45:2469–2475
  • Purpose: To enhance clinician’s decision-making by diagnosing hepatocellular carcinoma (HCC) in cirrhotic patients with indeterminate liver nodules using quantitative imaging features extracted from triphasic CT scans.
    Conclusion: A signature using a single feature was validated in a multicenter retrospective cohort to diagnose HCC in cirrhotic patients with indeterminate liver nodules. Artificial intelligence could enhance clinicians’ decision by identifying a subgroup of patients with high HCC risk.
    Radiomics machine-learning signature for diagnosis of hepatocellular carcinoma in cirrhotic patients with indeterminate liver nodules
    Fatima-Zohra Mokrane et al.
    European Radiology (2020) 30:558–570
  • "Our study assembled the largest radiomics dataset of indeterminate cirrhotic liver nodules to date and offers a proof of concept that machine-learning-based radiomics signature using change in quantitative CT features across the arterial and portal venous phases can allow a non-invasive accurate diagnosis of HCCs in cirrhotic patients with indeterminate nodules. This signature would allow for identification of high HCC–risk patients, who should be prioritized for therapy, allowing thus clinically significant benefits.”
    Radiomics machine-learning signature for diagnosis of hepatocellular carcinoma in cirrhotic patients with indeterminate liver nodules
    Fatima-Zohra Mokrane et al.
    European Radiology (2020) 30:558–570
  • Results: The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonize calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.
    Conclusion: IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reli- ability and corresponding performance of prognostic models in radiomics.
    Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform
    Isabella Fornacon-Wood et al.
    European Radiology https://doi.org/10.1007/s0330-020-06957-9
  • • Reliability of radiomic features varies between feature calculation platforms and with choice of software version.
    • Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised.
    • IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features.
    Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform
    Isabella Fornacon-Wood et al.
    European Radiology https://doi.org/10.1007/s0330-020-06957-9 
  • Purpose: The purpose of this study was to determine whether computed tomography (CT)-based machine learning of radiomics features could help distinguish autoimmune pancreatitis (AIP) from pancreatic ductal adenocarcinoma (PDAC).
    Results: The pancreas was diffusely involved in 37 (37/89; 41.6%) patients with AIP and not diffusely in 52(52/89; 58.4%) patients. Using machine learning, 95.2% (59/62; 95% confidence interval [CI]: 89.8–100%),83.9% (52:67; 95% CI: 74.7–93.0%) and 77.4% (48/62; 95% CI: 67.0–87.8%) of the 62 test patients werecorrectly classified as either having PDAC or AIP with thin-slice venous phase, thin-slice arterial phase, and thick-slice venous phase CT, respectively. Three of the 29 patients with AIP (3/29; 10.3%) were incorrectly classified as having PDAC but all 33 patients with PDAC (33/33; 100%) were correctly classified with thin-slice venous phase with 89.7% sensitivity (26/29; 95% CI: 78.6–100%) and 100% specificity (33/33;95% CI: 93–100%) for the diagnosis of AIP, 95.2% accuracy (59/62; 95% CI: 89.8–100%) and area under the curve of 0.975 (95% CI: 0.936–1.0).
    Conclusions: Radiomic features help differentiate AIP from PDAC with an overall accuracy of 95.2%.
  • Purpose: The purpose of this study was to determine whether computed tomography (CT)-based machine learning of radiomics features could help distinguish autoimmune pancreatitis (AIP) from pancreatic ductal adenocarcinoma (PDAC).
    Conclusions: Radiomic features help differentiate AIP from PDAC with an overall accuracy of 95.2%.
    Differentiating autoimmune pancreatitis from pancreatic ductal adenocarcinoma with CT radiomics features
    S. Park, L.C. Chu, R.H. Hruban, Vogelstein, K.W. Kinzler, A.L. Yuille, Fouladi, S. Shayesteh, S. Ghandili, C.L. Wolfgang, R. Burkhart, J. He, E.K. Fishman, S. Kawamoto
    Diagnostic and Interventional Imaging (in press)
  • •CT radiomics differentiates AIP from PDAC with 89.7% sensitivity and 100% specificity.
    •Thin slice CT radiomics better differentiates AIP from PDAC than thick slice CT radiomics.
    •Venous phase CT radiomics better differentiates AIP from PDAC than arterial phase radiomics.
    Differentiating autoimmune pancreatitis from pancreatic ductal adenocarcinoma with CT radiomics features
    S. Park, L.C. Chu, R.H. Hruban, Vogelstein, K.W. Kinzler, A.L. Yuille, Fouladi, S. Shayesteh, S. Ghandili, C.L. Wolfgang, R. Burkhart, J. He, E.K. Fishman, S. Kawamoto
    Diagnostic and Interventional Imaging (in press)
  • “AIP has clinical and imaging features that overlap with those of pancreatic ductal adenocarcinoma (PDAC) and can pose a significant diagnostic dilemma even for experienced radiologists . The management of these two conditions is markedly different. Patients with AIP are initially treated with oral corticosteroids, while patients with PDAC are treated with a combination of surgical resection and chemotherapy. The most common presentation of AIP is obstructive jaundice and pancreatic enlargement, which mimics that of PDAC and 2–6% of patients undergoing surgical resection for suspected pancreatic cancer are actually diagnosed with AIP upon histopathological analysis. Computed tomography (CT) plays an important role in the evaluation of suspected pancreatic cancer, and is often the initial diagnostic imaging modality. It is of utmost importance to correctly differentiate AIP from PDAC early in the disease process so as to administer the proper treatment and avoid unnecessary pancreatic resections in patients with AIP.”
    Differentiating autoimmune pancreatitis from pancreatic ductal adenocarcinoma with CT radiomics features
    S. Park, L.C. Chu, R.H. Hruban, Vogelstein, K.W. Kinzler, A.L. Yuille, Fouladi, S. Shayesteh, S. Ghandili, C.L. Wolfgang, R. Burkhart, J. He, E.K. Fishman, S. Kawamoto
    Diagnostic and Interventional Imaging (in press)

  • Differentiating autoimmune pancreatitis from pancreatic ductal adenocarcinoma with CT radiomics features
    S. Park, L.C. Chu, R.H. Hruban, Vogelstein, K.W. Kinzler, A.L. Yuille, Fouladi, S. Shayesteh, S. Ghandili, C.L. Wolfgang, R. Burkhart, J. He, E.K. Fishman, S. Kawamoto
    Diagnostic and Interventional Imaging (in press)
  • "In conclusion, radiomics analysis of CT images is reasonably accurate in differentiating AIP from PDAC. Using such features, in combination with clinical and standard radiologic analyses, may improve the accuracy of AID diagnosis and spare patients’ unnecessary surgical procedure.”
    Differentiating autoimmune pancreatitis from pancreatic ductal adenocarcinoma with CT radiomics features
    S. Park, L.C. Chu, R.H. Hruban, Vogelstein, K.W. Kinzler, A.L. Yuille, Fouladi, S. Shayesteh, S. Ghandili, C.L. Wolfgang, R. Burkhart, J. He, E.K. Fishman, S. Kawamoto
    Diagnostic and Interventional Imaging (in press)
  • "Our results showed that by combining radiomics features, AIP could be distinguished from PDAC with a sensitivity of 89.7% and a specificity of 100%, and an overall accuracy of 95.2%. Among 3 patients with focal AIP were falsely classified as PDAC using radiomics features, two patients had focal AIP in the head with a plastic stent in the common bile duct, which can sensitively affect to the quantitative feature computation. In our study, the accuracy was higher than that in a previous study that evaluated CT to differentiate AIP from PDAC based on morphological features. In that study, the mean accuracies for diagnosing AIP and PDAC were 68% and 83%, respectively. In our study, AIP was considered as a diagnosis or differential diagnosis by the radiologists in only in 67% of patients with AIP not already suspected to be AIP at the time of CT examination.”
    Differentiating autoimmune pancreatitis from pancreatic ductal adenocarcinoma with CT radiomics features
    S. Park, L.C. Chu, R.H. Hruban, Vogelstein, K.W. Kinzler, A.L. Yuille, Fouladi, S. Shayesteh, S. Ghandili, C.L. Wolfgang, R. Burkhart, J. He, E.K. Fishman, S. Kawamoto
    Diagnostic and Interventional Imaging (in press)
  • “We found that radiomics features were better at distinguishing AIP from PDAC using venous phase CT images than using arterial phase images. We also performed radiomics analysis on both thin- and thick-slice reconstructions. We found that thin-slice CT based radiomics signature had better diagnostic performance than thick-slice, as reported in pulmonary nodules and lung cancer in prior studies.”
    Differentiating autoimmune pancreatitis from pancreatic ductal adenocarcinoma with CT radiomics features
    S. Park, L.C. Chu, R.H. Hruban, Vogelstein, K.W. Kinzler, A.L. Yuille, Fouladi, S. Shayesteh, S. Ghandili, C.L. Wolfgang, R. Burkhart, J. He, E.K. Fishman, S. Kawamoto
    Diagnostic and Interventional Imaging (in press)
  • Purpose: The purpose of this study is to evaluate diagnostic performance of a commercially available radiomics research prototype vs. an in-house radiomics software in the binary classification of CT images from patients with pancreatic ductal adenocarcinoma (PDAC) vs. healthy controls.
    Conclusion: Commercially available and in-house radiomics software achieve similar diagnostic performance, which may lower the barrier of entry for radiomics research and allow more clinician-scientists to perform radiomics research.
    Diagnostic performance of commercially available vs. in‐house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls
    Linda C. Chu · Berkan Solmaz · Seyoun Park · Satomi Kawamoto · Alan L. Yuille · Ralph H. Hruban · Elliot K. Fishman
    Abdominal Radiology https://doi.org/10.1007/s00261-020-02556-w
  • “Results: When 40 radiomics features were used in the random forest classification, in-house software achieved superior sensitivity (1.00) and accuracy (0.992) compared to the commercially available research prototype (sensitivity = 0.950, accuracy = 0.968). When the number of features was reduced to five features, diagnostic performance of the in-house soft- ware decreased to sensitivity (0.950), specificity (0.923), and accuracy (0.936). Diagnostic performance of the commercially available research prototype was unchanged.”
    Diagnostic performance of commercially available vs. in‐house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls
    Linda C. Chu · Berkan Solmaz · Seyoun Park · Satomi Kawamoto · Alan L. Yuille · Ralph H. Hruban · Elliot K. Fishman
    Abdominal Radiology https://doi.org/10.1007/s00261-020-02556-w
  • “Radiomics has the potential to generate imaging biomarkers for classification and prognostication. Technical parameters from image acquisition to feature extraction and analysis have the potential to affect radiomics features. The current study used the same CT images with manual segmentation on both a commercially available research prototype and in-house radiomics software to control for any variability at the image acquisition step and compared the diagnostic performance of the two programs. Both programs achieved similar diagnostic performance in the binary classification of CT images from patients with PDAC and healthy control subjects, despite differences in the radiomics fea-tures they employed (854 features in commercial program vs. 478 features in in-house program).”
    Diagnostic performance of commercially available vs. in‐house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls
    Linda C. Chu · Berkan Solmaz · Seyoun Park · Satomi Kawamoto · Alan L. Yuille · Ralph H. Hruban · Elliot K. Fishman
    Abdominal Radiology https://doi.org/101007/s00261-020-02556-w
  • "This is reassuring that even though there may be variations in the computed values for radiomics features, the differences do not seem to significantly impact the overall diagnostic performance of the constellation of radiomics features. This is important for the broader implementation of radiomics research. Currently, many radiomics studies have been performed using proprietary in-house software, which requires in-house expertise in computer science, a luxury that only a few academic centers can afford. The results of this study show that commercially available radiomics software may be a viable alternative to in-house computer science expertise, which can lower the barrier of entry for radiomics research and allow clinicians to validate findings of the published studies with their own local datasets.”
    Diagnostic performance of commercially available vs. in‐house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls
    Linda C. Chu · Berkan Solmaz · Seyoun Park · Satomi Kawamoto · Alan L. Yuille · Ralph H. Hruban · Elliot K. Fishman
    Abdominal Radiology https://doi.org/101007/s00261-020-02556-w
  • “This study showed that a commercially available radiomics software may be able to achieve similar diagnostic performance as an in-house radiomics software. The results obtained from one radiomics software may be transferrable to another system. Availability of commercial radiom ics software may lower the barrier of entry for radiomics research and allow more researchers to engage in this exciting area of research.”
    Diagnostic performance of commercially available vs. in‐house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls
    Linda C. Chu · Berkan Solmaz · Seyoun Park · Satomi Kawamoto · Alan L. Yuille · Ralph H. Hruban · Elliot K. Fishman
    Abdominal Radiology https://doi.org/101007/s00261-020-02556-w 

  • Assessing Radiology Research on Artificial Intelligence:
    A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • “As an interim step, the Radiology editorial board has developed a list of nine key considerations that help us evaluate AI research (Table). The goal of these considerations is to improve the soundness and applicability of AI research in diagnostic imaging. These considerations are enumerated for the authors, but manuscript reviewers and readers may also find these points to be helpful.”
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • “1. Carefully define all three image sets (training, validation, and test sets of images) of the AI experiment. As sum- marized by Park and Han, the AI algorithm is trained on an initial set of images according to a standard of reference. The training algorithm is tuned and validated on a separate set of im- ages. Finally, an independent “test” set of images is used to report final statistical results of the AI. Ideally, each of the three sets of images should be independent, without overlap. Also, the inclusion and exclusion criteria for the dataset, in addition to the justification for removing any outlier, should be explained.”
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • “ 2. Use an external test set for final statistical reporting. ML/AI models are very prone to overfitting, meaning that they work well only for images on which they were trained. Ideally, an outside set of images (eg, from another institution, the external test set) is used for final assessment to determine if the ML/AI model will generalize.”
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • “3. Use multivendor images, preferably for each phase of the AI evaluation (training, validation, test sets). Radiologists are aware that MRI scans from one vendor do not look like those from another vendor. Such differences are detected by radiomics and AI algorithms. Vendor-specific algorithms are of much less interest than multivendor AI algorithms.”
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 4. Justify the size of the training, validation, and test sets. The number of images required to train an AI algorithm depends on the application. For example, an AI model may learn image segmentation after only a few hundred images, while thousands of chest radiographs may simultaneously be needed to detect lung nodules or multiple abnormalities. In their work classifying chest radiographs as normal or abnormal, Dunnmon et al began with 200000 chest images; however, their AI algo- rithm showed little benefit for improved performance after just 20000 chest radiographs. For many applications, the “correct” number of images may be unknown at the start of the research. The research team should evaluate the relationship between the number of training images versus model performance. For the test set, traditional sample size statistical considerations can be applied to determine the minimum number of images needed.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 5. Train the AI algorithm using a standard of reference that is widely accepted in our field. For chest radiographs, a panel of expert radiologists interpreting the chest radiograph is an inferior standard of reference compared with the chest CT. Similarly, the radiology report is considered an inferior standard of reference relative to dedicated “research readings” of the chest CT scans. Although surprising to nonradiologists, this journal and other high-impact journals in our field do not consider the clinical report to be a high-quality standard of reference for any research study in our field, including AI. Clinical reports often have nuanced conclusions and are generated for patient care and not for research purposes. For instance, degenerative spine disease may have little significance at 80 years old but could be critical at age 15.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 6. Describe any preparation of images for the AI algorithm. For coronary artery disease on CT angiograms, did the AI interpret all 300 source images? Or did the authors manu- ally select relevant images or crop images to a small field of view around the heart? Such preparation and annotation of images greatly affects radiologist understanding of the AI model. Manual cropping of tumor features is standard in radiomics studies; such studies should always evaluate the relationship of the size and reproducibility of the cropped volume to the final statistical result.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 7. Benchmark the AI performance to radiology experts. For computer scientists working on AI, competitions and leader boards for the “best” AI are common. Results frequently com- pare one AI to another based on the area under the receiver op- erating characteristic curve (AUC). However, to treat a patient, physicians are much more interested in the comparison of the AI algorithm to expert readers but not just any readers. Experienced radiologist readers are preferred to benchmark an algorithm de- signed to detect radiologic abnormalities. For example, when evaluating an AI algorithm to detect stroke on CT scans, expert neuroradiologists (rather than generalists or neurologists) are known to have the highest performance.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 8. Demonstrate how the AI algorithm makes decisions. As indicated above, computer scientists conducting imaging research often summarize their results as a single AUC value. That AUC is compared with the competitor, the prior best al- gorithm. Unfortunately, the AUC value alone has little relation- ship to clinical medicine. Even a high AUC value of 0.95 may include an operating mode where 99 of 100 abnormalities are missed. To help clinicians understand the AI performance, many research teams overlay colored probability maps from the AI on the source images.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 9. The AI algorithm should be publicly available so that claims of performance can be verified. Just like MRI or CT scanners, AI algorithms need independent validation. Commercial AI products may work in the computer laboratory but have poor function in the reading room. “Trust but verify” is essential for AI that may ultimately be used to help prescribe therapy for our patients. All AI algorithms should be made publicly available via a website such as GitHub. Commercially available algorithms are considered publicly available.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515 

  • Assessing Radiology Research on Artificial Intelligence:
    A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • “As an interim step, the Radiology editorial board has developed a list of nine key considerations that help us evaluate AI research (Table). The goal of these considerations is to improve the soundness and applicability of AI research in diagnostic imaging. These considerations are enumerated for the authors, but manuscript reviewers and readers may also find these points to be helpful.”
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • “1. Carefully define all three image sets (training, validation, and test sets of images) of the AI experiment. As sum- marized by Park and Han, the AI algorithm is trained on an initial set of images according to a standard of reference. The training algorithm is tuned and validated on a separate set of im- ages. Finally, an independent “test” set of images is used to report final statistical results of the AI. Ideally, each of the three sets of images should be independent, without overlap. Also, the inclusion and exclusion criteria for the dataset, in addition to the justification for removing any outlier, should be explained.”
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • “ 2. Use an external test set for final statistical reporting. ML/AI models are very prone to overfitting, meaning that they work well only for images on which they were trained. Ideally, an outside set of images (eg, from another institution, the external test set) is used for final assessment to determine if the ML/AI model will generalize.”
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • “3. Use multivendor images, preferably for each phase of the AI evaluation (training, validation, test sets). Radiologists are aware that MRI scans from one vendor do not look like those from another vendor. Such differences are detected by radiomics and AI algorithms. Vendor-specific algorithms are of much less interest than multivendor AI algorithms.”
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 4. Justify the size of the training, validation, and test sets. The number of images required to train an AI algorithm depends on the application. For example, an AI model may learn image segmentation after only a few hundred images, while thousands of chest radiographs may simultaneously be needed to detect lung nodules or multiple abnormalities. In their work classifying chest radiographs as normal or abnormal, Dunnmon et al began with 200000 chest images; however, their AI algo- rithm showed little benefit for improved performance after just 20000 chest radiographs. For many applications, the “correct” number of images may be unknown at the start of the research. The research team should evaluate the relationship between the number of training images versus model performance. For the test set, traditional sample size statistical considerations can be applied to determine the minimum number of images needed.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 5. Train the AI algorithm using a standard of reference that is widely accepted in our field. For chest radiographs, a panel of expert radiologists interpreting the chest radiograph is an inferior standard of reference compared with the chest CT. Similarly, the radiology report is considered an inferior standard of reference relative to dedicated “research readings” of the chest CT scans. Although surprising to nonradiologists, this journal and other high-impact journals in our field do not consider the clinical report to be a high-quality standard of reference for any research study in our field, including AI. Clinical reports often have nuanced conclusions and are generated for patient care and not for research purposes. For instance, degenerative spine disease may have little significance at 80 years old but could be critical at age 15.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 6. Describe any preparation of images for the AI algorithm. For coronary artery disease on CT angiograms, did the AI interpret all 300 source images? Or did the authors manu- ally select relevant images or crop images to a small field of view around the heart? Such preparation and annotation of images greatly affects radiologist understanding of the AI model. Manual cropping of tumor features is standard in radiomics studies; such studies should always evaluate the relationship of the size and reproducibility of the cropped volume to the final statistical result.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 7. Benchmark the AI performance to radiology experts. For computer scientists working on AI, competitions and leader boards for the “best” AI are common. Results frequently com- pare one AI to another based on the area under the receiver op- erating characteristic curve (AUC). However, to treat a patient, physicians are much more interested in the comparison of the AI algorithm to expert readers but not just any readers. Experienced radiologist readers are preferred to benchmark an algorithm de- signed to detect radiologic abnormalities. For example, when evaluating an AI algorithm to detect stroke on CT scans, expert neuroradiologists (rather than generalists or neurologists) are known to have the highest performance.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 8. Demonstrate how the AI algorithm makes decisions. As indicated above, computer scientists conducting imaging research often summarize their results as a single AUC value. That AUC is compared with the competitor, the prior best al- gorithm. Unfortunately, the AUC value alone has little relation- ship to clinical medicine. Even a high AUC value of 0.95 may include an operating mode where 99 of 100 abnormalities are missed. To help clinicians understand the AI performance, many research teams overlay colored probability maps from the AI on the source images.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515
  • 9. The AI algorithm should be publicly available so that claims of performance can be verified. Just like MRI or CT scanners, AI algorithms need independent validation. Commercial AI products may work in the computer laboratory but have poor function in the reading room. “Trust but verify” is essential for AI that may ultimately be used to help prescribe therapy for our patients. All AI algorithms should be made publicly available via a website such as GitHub. Commercially available algorithms are considered publicly available.
    Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers—From the Radiology Editorial Board
    David A.Bluemke et al.
    Radiology 2019; (in press) https://doi.org/10.1148/radiol.2019192515 

  • State‐of‐the‐art in radiomics of hepatocellular carcinoma: a review of basic principles, applications, and limitations
    Santos JM et al.
    Abdominal Radiology 2019 (in press)
  • “Radiomics is characterized by the extraction of quantitative imaging features from conventional imaging modalities using computer based algorithms and the correlation of these features with relevant clinical endpoints, such as pathology, therapeutic response, and survival. These quantitative data are called radiomics features, of which texture features are a subset.”
    State‐of‐the‐art in radiomics of hepatocellular carcinoma: a review of basic principles, applications, and limitations
    Santos JM et al.
    Abdominal Radiology 2019 (in press)
  • Radiomics is an emerging field that converts medical imaging into high‐dimensional mineable features, providing a quantitative assessment of the image. These features can then be associated to clinical endpoints, such as pathology, therapeutic response, and survival. With the quantitative analysis of digital imaging, radiomics can potentially detect specific characteristics of a disease that otherwise could not be accessed visually with a potential to inform future precision medicine.”
    State‐of‐the‐art in radiomics of hepatocellular carcinoma: a review of basic principles, applications, and limitations
    Santos JM et al.
    Abdominal Radiology 2019 (in press)
  • “Radiomics is a new field in medical imaging with the potential of changing medical practice. Radiomics is characterized by the extraction of several quantitative imaging features which are not visible to the naked eye from conventional imaging modalities, and its correlation with specific relevant clinical endpoints, such as pathology, therapeutic response, and survival. Several studies have evaluated the use of radiomics in patients with hepatocellular carcinoma (HCC) with encouraging results, particularly in the pretreatment prediction of tumor biological characteristics, risk of recurrence, and survival.”
    State‐of‐the‐art in radiomics of hepatocellular carcinoma: a review of basic principles, applications, and limitations
    Santos JM et al.
    Abdominal Radiology 2019 (in press)
  • OBJECTIVE. The purpose of this study was to systematically review the radiomics literature on renal mass characterization in terms of reproducibility and validation strategies.
    CONCLUSION. Workflow characteristics described in the radiomics literature about renal mass characterization are heterogeneous. To bring radiomics from a mere research area to clinical use, the field needs many more papers that consider the reproducibility of radiomic features and include independent or external validation in their workflow.
    Radiomics of Renal Masses: Systematic Review of Reproducibility and Validation Strategies
    Kocak B et al.
    AJR 2020; 214:129–136
  • “Fifth, regarding segmentation styles, the majority of the studies were conducted with single-slice without multiple sampling or 3D approaches with almost similar rates. It is noteworthy that some authors used segmen- tation as a data augmentation technique to increase and balance the number of labeled slices, which in this study we call the single- slice with multiple sampling style.”
    Radiomics of Renal Masses: Systematic Review of Reproducibility and Validation Strategies
    Kocak B et al.
    AJR 2020; 214:129–136
  • "In this study, we systematically reviewed the radiomics literature about renal mass characterization with a particular focus on two important methodologic quality issues: feature reproducibility and model validation strategies. Our qualitative synthesis showed that these strategies varied to a large extent. Despite the well-known reproducibility problem of radiomics, approximately one-half of the papers had no reproducibility analysis in their workflow.”
    Radiomics of Renal Masses: Systematic Review of Reproducibility and Validation Strategies
    Kocak B et al.
    AJR 2020; 214:129–136
  • "To bring the field of radiomics of renal masses from mere research to the clinical stage, future research should be designed with independent or external validation. The systematic work conducted for this study should provide guidance for researchers and reviewers in this ever-developing research area about what has been done to date and what needs to be done for the future of this realm.”
    Radiomics of Renal Masses: Systematic Review of Reproducibility and Validation Strategies
    Kocak B et al.
    AJR 2020; 214:129–136
  • Background: Texture analysis of medical images has been reported to be a reliable method for differential diagnosis of neoplasms. This study was to investigate the performance of textural features and the combined performance of textural features and morphological characteristics in the differential diagnosis of pancreatic serous and mucinous cystadenomas.
    Conclusions: In conclusion, our preliminary results highlighted the potential of CT texture analysis in discriminating pancreatic serous cystadenoma from mucinous cystadenoma. Furthermore, the combination of morphological characteristics and textural features can significantly improve the diagnostic performance, which may provide a reliable method for selecting patients with surgical intervention indications in consideration of the different treatment principles of the two diseases.
    Differential diagnosis of pancreatic serous cystadenoma and mucinous cystadenoma: utility of textural features in combination with morphological characteristics
    Jing Yang et al.
    BMC Cancer (2019) 19:1223 https://doi.org/10.1186/s12885-019-6421-7
  • “In conclusion, our preliminary results highlighted the potential of CT texture analysis to discriminate pancreatic serous cystadenoma and mucinous cystadenoma. Furthermore, the combination of morphological characteristics and textural features can significantly improve differential diagnostic performance, which may provide a reliable method for selecting pancreatic cystadenoma patients who need surgical intervention.”
    Differential diagnosis of pancreatic serous cystadenoma and mucinous cystadenoma: utility of textural features in combination with morphological characteristics
    Jing Yang et al.
    BMC Cancer (2019) 19:1223 https://doi.org/10.1186/s12885-019-6421-7
  • "Thus, surgical intervention should be proposed in a minority of patients with serous cystadenoma, and only for those who had uncertain diagnosis after systemic examinations or had symptoms. Given the risk of invasive disease and the relatively young age at diagnosis, surgical management is recommended for all mucinous cystadenoma patients who are medically fit for the surgery. Therefore, the differential diagnosis of the two diseases is clinically crucial for the choice of treatment regimen.”
    Differential diagnosis of pancreatic serous cystadenoma and mucinous cystadenoma: utility of textural features in combination with morphological characteristics
    Jing Yang et al.
    BMC Cancer (2019) 19:1223 https://doi.org/10.1186/s12885-019-6421-7 
  • Radiomics: extraction of features from diagnostic images, the final product of which is a quantitative feature/parameter, measurable and mineable from images. A Radiomics analysis can extract over 400 features from a region of interest in a CT, MRI, or PET study, and correlate these features with each other and other data, far beyond the capability of the human eye or brain to appreciate. Such features may be used to predict prognosis and response to treatment . AI can support analysis of radiomics features and help in the correlation between radiomics and other data (proteomics, genomics, liquid biopsy, etc.) by building patients’ signatures.
    What the radiologist should know about artificial intelligence – an ESR white paper
    Insights into Imaging (2019) 10:44 https://doi.org/10.1186/s13244-019-0738-2 
  • Radiomics: extraction of features from diagnostic images, the final product of which is a quantitative feature/parameter, measurable and mineable from images. A Radiomics analysis can extract over 400 features from a region of interest in a CT, MRI, or PET study, and correlate these features with each other and other data, far beyond the capability of the human eye or brain to appreciate. Such features may be used to predict prognosis and response to treatment . AI can support analysis of radiomics features and help in the correlation between radiomics and other data (proteomics, genomics, liquid biopsy, etc.) by building patients’ signatures.
    What the radiologist should know about artificial intelligence – an ESR white paper
    Insights into Imaging (2019) 10:44 https://doi.org/10.1186/s13244-019-0738-2 
  • “Results: Radiomics features extracted from contrast-enhanced CT were able to discriminate pancreatic mucinous cystadenomas from serous cystadenomas in both the training group (slice thickness of 2 mm, AUC 0.77, sensitivity 0.95, specificity 0.83, accuracy 0.85; slice thickness of 5 mm, AUC 0.72, sensitivity 0.90, specificity 0.84, accuracy 0.86) and the validation group (slice thickness of 2 mm, AUC 0.66, sensitivity 0.86, specificity 0.71, accuracy 0.74; slice thickness of 5 mm, AUC 0.75, sensitivity 0.85, specificity 0.83, accuracy 0.83).
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • The parameters derived from texture analysis and several clinicopathological characteristic (age, gender, size, location of lesions, enhancement of peripheral wall, mural nodules, and calcification of lesions) were analyzed using random forest and Least Absolute Shrinkage and Selection Operator (LASSO) methods. In the group of 2 mm slice thickness, 22 parameters were obtained using the random forest analysis and 12 parameters were obtained using LASSO method; 5 overlapping parameters were discovered. In the group of 5 mm slice thickness, 18 parameters were obtained using the random forest analysis and 14 parameters were obtained using LASSO method; 4 overlapping parameters were discovered. Those selected textural parameters were given as mean ± standard deviation. Statistical differences of textural parameters were analyzed using independent sample t-test. A p-value of <0.05 was considered significant.
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • Given the benign nature of pancreatic serous cystadenomas and malignant potential of mucinous cystadenomas, resection is not suggested for most of the patients with serous cystadenoma while surgical treatment is recommended for all surgical fit patients with mucinous cystadenoma . Therefore, preoperative differential diagnosis is critical. Currently, cross- sectional imaging, endoscopic ultrasound (EUS), fine-needle aspiration (FNA) biopsy and cyst fluid analysis were frequently employed to assist in the differential diagnosis. EUS with cyst fluid analysis is the most important mean to distinguish pancreatic mucinous cystadenomas from serous cystadenomas. A cyst fluid carcinoembryonic antigen (CEA) level > 192 ng/mL has been reported to be useful for identification of mucinous cystadenomas, with a sensitivity of 73% and specificity of 84%. However, cyst fluid analysis is limited by its invasiveness
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • In general, the heterogeneity of tissue is composed of multiple texture patterns, so a single textural parameter cannot fully display the gross textural characteristics of tumor. In the preliminary analysis of this study, we also tried to analyze individual factors, and the results were not satisfactory. In consideration of this, a complex of integrated different textural parameters is required to represent gross texture of tumor more comprehensively. Random forest model, a powerful machine-learning approach, has proved successful in classifying subjects into the correct group. Previous studies have also indicated that random forest model could be used in the analysis of textural features. In this study, random forest model was able to discriminate between pancreatic mucinous cystadenomas and serous cystadenomas.
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • In conclusion, our study provided preliminary evidence that analysis texture of lesions in CT images was a reliable method to differentiate diagnosis of pancreatic mucinous cystadenomas and serous cystadenomas, which may provide a convenient, non-invasive and repeatable approach to determine whether surgery is needed in clinical practice. However, multicentre studies with larger sample size are needed to confirm these results.
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • “Results: Radiomics features extracted from contrast-enhanced CT were able to discriminate pancreatic mucinous cystadenomas from serous cystadenomas in both the training group (slice thickness of 2 mm, AUC 0.77, sensitivity 0.95, specificity 0.83, accuracy 0.85; slice thickness of 5 mm, AUC 0.72, sensitivity 0.90, specificity 0.84, accuracy 0.86) and the validation group (slice thickness of 2 mm, AUC 0.66, sensitivity 0.86, specificity 0.71, accuracy 0.74; slice thickness of 5 mm, AUC 0.75, sensitivity 0.85, specificity 0.83, accuracy 0.83).
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • The parameters derived from texture analysis and several clinicopathological characteristic (age, gender, size, location of lesions, enhancement of peripheral wall, mural nodules, and calcification of lesions) were analyzed using random forest and Least Absolute Shrinkage and Selection Operator (LASSO) methods. In the group of 2 mm slice thickness, 22 parameters were obtained using the random forest analysis and 12 parameters were obtained using LASSO method; 5 overlapping parameters were discovered. In the group of 5 mm slice thickness, 18 parameters were obtained using the random forest analysis and 14 parameters were obtained using LASSO method; 4 overlapping parameters were discovered. Those selected textural parameters were given as mean ± standard deviation. Statistical differences of textural parameters were analyzed using independent sample t-test. A p-value of <0.05 was considered significant.
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • Given the benign nature of pancreatic serous cystadenomas and malignant potential of mucinous cystadenomas, resection is not suggested for most of the patients with serous cystadenoma while surgical treatment is recommended for all surgical fit patients with mucinous cystadenoma . Therefore, preoperative differential diagnosis is critical. Currently, cross- sectional imaging, endoscopic ultrasound (EUS), fine-needle aspiration (FNA) biopsy and cyst fluid analysis were frequently employed to assist in the differential diagnosis. EUS with cyst fluid analysis is the most important mean to distinguish pancreatic mucinous cystadenomas from serous cystadenomas. A cyst fluid carcinoembryonic antigen (CEA) level > 192 ng/mL has been reported to be useful for identification of mucinous cystadenomas, with a sensitivity of 73% and specificity of 84%. However, cyst fluid analysis is limited by its invasiveness
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • In general, the heterogeneity of tissue is composed of multiple texture patterns, so a single textural parameter cannot fully display the gross textural characteristics of tumor. In the preliminary analysis of this study, we also tried to analyze individual factors, and the results were not satisfactory. In consideration of this, a complex of integrated different textural parameters is required to represent gross texture of tumor more comprehensively. Random forest model, a powerful machine-learning approach, has proved successful in classifying subjects into the correct group. Previous studies have also indicated that random forest model could be used in the analysis of textural features. In this study, random forest model was able to discriminate between pancreatic mucinous cystadenomas and serous cystadenomas.
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • In conclusion, our study provided preliminary evidence that analysis texture of lesions in CT images was a reliable method to differentiate diagnosis of pancreatic mucinous cystadenomas and serous cystadenomas, which may provide a convenient, non-invasive and repeatable approach to determine whether surgery is needed in clinical practice. However, multicentre studies with larger sample size are needed to confirm these results.
    Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning
    Yang J et al.
    Front. Oncol., 12 June 2019 /doi.org/10.3389/fonc.2019.00494
  • “Texture analysis has a potential role in distinguishing benign from malignant adrenal nodules on CECT and may decrease the need for additional imaging studies in the workup of incidentally discovered adrenal nodules.”
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • “Current imaging methods can diagnose lipid-rich adenomas with the use of either unenhanced CT or chemical-shift MRI and can diagnose lipid-poor adenomas on the basis of calculation of the percentage washout on contrast-enhanced CT (CECT).”
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • “Image-based texture analysis is a quantitative technique that provides a measure of lesion heterogeneity on the basis of local variations in image brightness. First-order statistics- based texture analysis evaluates the number of pixels that have a particular gray-level value within a defined ROI. First-order texture analysis does not account for the location of the pixels within the ROI. Second-order statistics- based texture analysis evaluates the location and spatial interrelationship s between pixels of variable gray-level values.”
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • For example, first-order texture analysis can determine how many pixels have attenuation of 0 HU within an adrenal nodule. Second-order texture analysis can determine whether those pixels with an attenuation of 0 HU within an adrenal nodule are distributed evenly or are clustered in groups.
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • “Texture analysis of CECT images showed higher diagnostic performance for the diagnosis of malignancy, compared with CECT attenuation. The performance of select individual CECT texture features (long-run high gray-level emphasis, entropy, and short-run low gray-level emphasis) were comparable to unenhanced attenuation on CT and the SII on MRI, which are the standard diagnostic imaging tests used to distinguish adrenal adenomas from metastases in clinical practice.”
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • Increased tumor heterogeneity is the most likely reason for the ability of texture analysis to predict adrenal malignancy on CECT. As is seen in Figure 2, lipid-poor adenomas appeared homogeneous on CECT, compared with malignant lesions, which appeared heterogeneous. We speculate that the administration of contrast material may make lipid- poor adenomas appear more homogeneous because both lipid-rich and lipid-poor areas will have uptake of contrast medium.
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • “Malignant adrenal lesions become more heterogeneous after contrast material administration because of tumor angiogenesis and increased conspicuity of tumor necrosis. In support of our theory, a recent study by Sasaguri et al. showed that adrenal metastases from renal carcinoma showed visibly higher internal heterogeneity, compared with benign adrenal masses on CECT.”
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • Another limitation of the present study is the retrospective nature of the data acquisition. Because this is an observational study, the type of scanner used for each patient was not controlled. One cannot underestimate the potential impact of variation in CT and MR image quality on the results of texture analysis. This factor alone represents a major challenge when one considers the robustness of applying texture analysis in the clinical setting.
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • “The introduction of radiomics has brought with it the vast expansion of the promise of quantitative and objective assessment of images. Interpretations are no longer limited to features like area, volume, and histogram-derived metrics; they can include hundreds of different features including shape, gray-level run-length matrices, Haralick texture, het- erogeneity, coarseness, or busyness.”
    How Can Radiomics Be Consistently Applied across Imagers and Institutions?
    Steiger P, Sood R
    Radiology (in press)
  • It allows for correction of radiomic measurements on the basis of their distribution and knowledge of covariates. The authors tested their method with one publicly available phantom data set and two patient data sets from patients with lung cancer. They convincingly showed that their method reduced im- ager-induced variability without sacrificing diagnostic sensitivity. Their article explains the method clearly and pro- vides all the references needed to replicate the work. This should encourage others to apply this method and test it in other radiomics studies and applications.
    How Can Radiomics Be Consistently Applied across Imagers and Institutions?
    Steiger P, Sood R
    Radiology (in press)
  • “In summary, the results of the present study indicate that the use of texture analysis for evaluation of adrenal nodules works best with CECT. This finding suggests that CT texture analysis may have a potential role in distinguishing benign lipid-poor ad- enomas from adrenal malignancy on single- phase CECT. Furthermore, the application of texture analysis may potentially decrease the need for additional imaging studies to workup incidentally discovered adrenal nodules.”
    Can Texture Analysis Be Used to Distinguish Benign From Malignant Adrenal Nodules on Unenhanced CT, Contrast-Enhanced CT, or In- Phase and Opposed-Phase MRI?
    Ho LM et al.
    AJR 2019; 212:554–561
  • “Radiomics holds the promise to become a tool at the disposal of the radiologist to expand the qualitative interpretation of the image, with additional quantitative information that can provide functional and prospective information not evident from the image alone. More studies are needed to fulfill this promise. The proposed algorithm has been shown to be effective in both thin- and thick-section CT images.”
    How Can Radiomics Be Consistently Applied across Imagers and Institutions?
    Steiger P, Sood R
    Radiology (in press)
  • “Therefore, the general success of radiomics in lung cancer and oncology will in part depend on the development and adoption of tailored image acquisition techniques for quantitative feature analysis. Radiomics will benefit from an extension of efforts already underway to standardize quantitative imaging, spearheaded by the Quantitative Imaging Biomarkers Alliance.”
    How Can Radiomics Be Consistently Applied across Imagers and Institutions?
    Steiger P, Sood R
    Radiology (in press)
  • ”Substantial hurdles remain until radiomics can become a routine tool in the radiology reading room of the future, as eloquently explained by Gillies et al. Among them is the need to validate any radiomics biomarkers in prospective multicenter studies. The variability introduced by the wide variety of avail- able equipment and imaging protocols must be controlled to allow these radiomic biomarkers to be used in a broader manner. The method presented by Orlhac et al. may have an important role in this research.”
    How Can Radiomics Be Consistently Applied across Imagers and Institutions?
    Steiger P, Sood R
    Radiology (in press)
  • “Substantial hurdles remain until radiomics can become a routine tool in the radiology reading room of the future, as eloquently explained by Gillies et al (1). Among them is the need to validate any radiomics biomarkers in prospective multicenter studies. The variability introduced by the wide variety of available equipment and imaging protocols must be controlled to allow these radiomic biomarkers to be used in a broader manner. The method presented by Orlhac et al (2) may have an important role in this research.”
    How Can Radiomics Be Consistently Applied across Imagers and Institutions?
    Peter Steiger, Rohit Sood
    Radiology 2019; 00:1–2
  • “The introduction of radiomics has brought with it the vast expansion of the promise of quantitative and objective assessment of images. Interpretations are no longer limited to features like area, volume, and histogram-derived metrics; they can include hundreds of different features including shape, gray-level run-length matrices, Haralick texture, heterogeneity, coarseness, or busyness (1). Putting such higher dimension image characteristics into the context of increasingly accessible clinical information about patients holds promise for evidence-based clinical decision support.”
    How Can Radiomics Be Consistently Applied across Imagers and Institutions?
    Peter Steiger, Rohit Sood
    Radiology 2019; 00:1–2
  • In this issue of Radiology, Orlhac et al (2) adapt a method originally used in genomics to correct variations in radiomic measurements caused by different imagers and imaging protocols (2). The proposed method is based on a statistical method called ComBat, which is readily available in the open-source R statistical programming language (R Foundation for Statistical Computing, Vienna, Austria). Unlike other previously published methods, this approach does not require images to be modified.
    How Can Radiomics Be Consistently Applied across Imagers and Institutions?
    Peter Steiger, Rohit Sood
    Radiology 2019; 00:1–2
  • Background: Radiomics extracts features from medical images more precisely and more accurately than visual assessment. However, radiomics features are affected by CT scanner parameters such as reconstruction kernel or section thickness, thus obscuring underlying biologically important texture features.
    Conclusion: Image compensation successfully realigned feature distributions computed from different CT imaging protocols and should facilitate multicenter radiomic studies.
    Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics
    Orlhac F et al.
    Radiology 2019; 00:1–7
  • “Radiomics extracts features from medical images that quantify tumor shape, intensity histogram, and texture of the lesions more precisely and more accurately than visual assessment by a radiologist to build models that involve features to assist patient treat- ment. In particular, texture analysis from CT images has led to promising results to distinguish between tumor lesions with different histopathologic characteristics and to predict treatment response or patient survival.”
    Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics
    Orlhac F et al.
    Radiology 2019; 00:1–7
  • Key points
    * Radiomic feature values obtained by using different CT imaging protocols or scanners can be corrected for the protocol or scanner effect by using the proposed compensation method.
    * The use of realigned features will enable multicentric studies to pool data from different sites to build reliable radiomic models based on large databases.
    * The proposed compensation method is easily available, fast, and requires neither phantom acquisition nor feature recalculation.
    Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics
    Orlhac F et al.
    Radiology 2019; 00:1–7
  • “Nonbiological differences related to CT scanner type can be removed from radiomic feature values, allowing radiomics features to be combined in multicenter or multivendor studies.”
    Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics
    Orlhac F et al.
    Radiology 2019; 00:1–7
  • “To correct for differences in features caused by the various imaging protocols, we used the ComBat function (https://github.com/Jfortin1/ComBatHarmonization) compensation method.”
    Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics
    Orlhac F et al.
    Radiology 2019; 00:1–7
  • “In conclusion, ComBat makes it possible to pool radiomic features from different CT protocols. This method appears promising to address the center effect in multicenter radiomic studies and to possibly raise the statistical power of those studies. ComBat is data driven, which means that the transformations identified by ComBat to set all data in a common space should be estimated for each study involving data from different cen- ters and protocols.”
    Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics
    Orlhac F et al.
    Radiology 2019; 00:1–7
  • “Moreover, radiomics-predicted lymph node metastasis emerged as a preoperative predictor of both disease-specific survival and recurrence-free survival after curative intent resection of biliary tract cancers (hazard ratios, 3.37 and 1.98, respectively). Overall, there was important personalized information for medical decision support.”
    CT-based Radiomics for Biliary Tract Cancer: A Possible Solution for Predicting Lymph Node Metastases
    Laghi A, Voena C
    Radiology 2019; 290:99–100
  • There are limitations. Although the model was built with rigorous methodologic structure, a multicentric study collecting a larger number of patients would be necessary to check for the generalizability of the radiomics signature. The influence of different CT parameters (eg, kilovolt, milliampere-seconds, and reconstruction filters) on extraction of radiomics features was not among the objectives of this study, although this is a relevant variable that might affect data consistency and limit the extensive use of the model.
    CT-based Radiomics for Biliary Tract Cancer: A Possible Solution for Predicting Lymph Node Metastases
    Laghi A, Voena C
    Radiology 2019; 290:99–100
  • A correlation with genomic profile of biliary tract cancers may have been desirable, especially in the era of target therapy where specific genomic profiles are associated with either response or resistance to a specific drug. Nevertheless, radiomics approaches seem to have a bright future, especially if collaborative multidisciplinary teams are involved. Ultimately, to achieve personalized medicine, personalized imaging must be involved.
    CT-based Radiomics for Biliary Tract Cancer: A Possible Solution for Predicting Lymph Node Metastases
    Laghi A, Voena C
    Radiology 2019; 290:99–100
  • “Ultimately, to achieve personalized medicine, personalized imaging must be involved."
    CT-based Radiomics for Biliary Tract Cancer: A Possible Solution for Predicting Lymph Node Metastases
    Laghi A, Voena C
    Radiology 2019; 290:99–100
  • “The further goal of radiomics analytics is to develop decision support tools, such as predictive models, by incorporating radiomics signature and other morphologic features. Radiomics models providing individualized risk estimation of LN metastasis have been developed and validated in studies focused on esophageal, colorectal, and bladder cancers with good results."
    CT-based Radiomics for Biliary Tract Cancer: A Possible Solution for Predicting Lymph Node Metastases
    Laghi A, Voena C
    Radiology 2019; 290:99–100
  • "Radiomics uses advanced image-processing techniques to extract a large number of quantitative parameters from imaging data, and its potential to improve diagnostic accuracy is increasingly being studied . Initial studies have reported promising performance of radiomics with and without the use of machine learning in the prediction of the prostate cancer Gleason score."
    Radiomic Machine Learning for Characterization of Prostate Lesions with MRI: Comparison to ADC Values
    Bonekamp D et al.
    Radiology 2018 (in press)
  • "In conclusion, this study compared the use of mean ADC and radiomics with machine learning for the characterization of lesions that were prospectively detected during routine clinical interpretation.
    Quantitative assessment of the mean ADC was more accurate than qualitative PI-RADS assessment in classifying a lesion as clinically significant prostate cancer. Radiomics provided additional data that ADC metrics (including mean ADC) were more valuable than other MRI features. In fact, at the current cohort size, no added benefit of the radiomic approach was found, and mean ADC is suggested as the best choice for quantitative prostate assessment."
    Radiomic Machine Learning for Characterization of Prostate Lesions with MRI: Comparison to ADC Values
    Bonekamp D et al.
    Radiology 2018 (in press)
  • Purpose: To determine the feasibility of using a deep learning approach to detect cartilage lesions (including cartilage softening, fibrillation, fissuring, focal defects, diffuse thinning due to cartilage degeneration, and acute cartilage injury) within the knee joint on MR images.
    Conclusion: is study demonstrated the feasibility of using a fully automated deep learning–based cartilage lesion detection system to evaluate the articular cartilage of the knee joint with high diagnostic performance and good intraobserver agreement for detecting cartilage degeneration and acute cartilage injury.
    Deep Learning Approach for Evaluating Knee MR Images: Achieving High Diagnostic Performance for Cartilage Lesion Detection
    FangLiu et al.
    Radiology 2018 (in press)
  • “Radiomics is a process that extracts a large number of quantitative features from medical images. It can potentially be applied to any medical condition, but it is currently applied mostly in oncology for quantification of tumour phenotype and for development of decision support tools. Deep learning and convolutional neural networks have the potential to automatically extract the significant features from images to help predict an important outcome (eg, cancer-specific mortality).”

    
Canadian Association of Radiologists White Paper 
on Artificial Intelligence in Radiology
An Tang et al. 
Canadian Association of Radiologists Journal 69 (2018) 120e135 

  • “With technological advances in computer science, it is anticipated that an increasing number of repetitive tasks will be automated over time. The PACS of all hospitals contain large imaging datasets with matching descriptions within radiology reports that can be used to perform ML on very large scale. The interactions between radiology images and their reports have been used to train ML for automated detection of disease in images [56]. Of note, a recent review of deep learning revealed that many recent applications in medical image analysis focus on 2D convolutional neural networks which do not directly leverage 3D information [57]. While 3D convolutional neural networks are emerging for analysis of multiplanar imaging (eg, CT), further research will be required to analyze multiparametric imaging examinations (eg, MRI).”


    Canadian Association of Radiologists White Paper 
on Artificial Intelligence in Radiology
An Tang et al. 
Canadian Association of Radiologists Journal 69 (2018) 120e135 

  • “AI techniques have been steadily developed since 1955 but recently have undergone a resurgence due to breakthrough performance arising from a combination of factors: wide availability of labeled data, advances in neural network architectures, and availability of parallel computing hardware. In radiology, AI applications currently focus on anomaly detection, segmentation, and classification of images. Familiarity with the terminology and key concepts in this field will allow the radiology community to critically analyze the opportunities, pitfalls, and challenges associated with the introduction of these new tools. Radiologists should become actively involved in research and development in collaboration with key stakeholders, scientists, and industrial partners to ensure radiologist oversight in the definition of use cases and validation process, and in the clinical application for patient care. Residency programs should integrate health informatics and computer science courses in AI in their curriculum.”


    Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology
An Tang et al. 
Canadian Association of Radiologists Journal 69 (2018) 120e135 

  • “Radiomics (or radiogenomics) is the correlation between the imaging appearance of cancer and the genomics of such. Advances in traditional machine learning and more novel deep learning approaches in this area have shown promising results. Moreover, deep learning techniques has achieved state-of-the-art results in biomedical image segmentation, which can be used to automatically segment and extract volumes of organs, specific tissues, and regions of interest. The radiology report of the future may automatically include such quantitative information, which could be used to assess disease and guide treatment decisions.”


    Machine Learning in Radiology: 
Applications Beyond Image Interpretation 
Paras Lakhani et al.
J Am Coll Radiol (in press)
  • “Ultimately, machine learning has the potential to dramatically improve patient care. Importantly for radiologists, machine learning algorithms can help address many problems in current-day radiology practices that do not involve image interpretation. Although much of the attention in the machine learning space has focused on the ability of machines to classify image findings, there are many other useful applications of machine learning that will improve efficiency and utilization of radiology practices today.”


    Machine Learning in Radiology: 
Applications Beyond Image Interpretation 
Paras Lakhani et al.
J Am Coll Radiol (in press)
© 1999-2020 Elliot K. Fishman, MD, FACR. All rights reserved.