Imaging Pearls ❯ Deep Learning ❯ Generative AI
|
-- OR -- |
|
- Generative artificial intelligence (AI) describes algorithms (such as ChatGPT) that can be used to create new content, including audio, code, images, text, simulations, and videos. Recent breakthroughs in the field have the potential to drastically change the way we approach content creation.
- It’s clear that generative AI tools like ChatGPT (the GPT stands for generative pretrained transformer) and image generator DALL-E (its name a mashup of the surrealist artist Salvador Dalí and the lovable Pixar robot WALL-E) have the potential to change how a range of jobs are performed. The full scope of that impact, though, is still unknown—as are the risks.
- Until recently, machine learning was largely limited to predictive models, used to observe and classify patterns in content. For example, a classic machine learning problem is to start with an image or several images of, say, adorable cats. The program would then identify patterns among the images, and then scrutinize random images for ones that would match the adorable cat pattern. Generative AI was a breakthrough. Rather than simply perceive and classify a photo of a cat, machine learning is now able to create an image or text description of a cat on demand.
- The outputs generative AI models produce may often sound extremely convincing. This is by design. But sometimes the information they generate is just plain wrong. Worse, sometimes it’s biased (because it’s built on the gender, racial, and myriad other biases of the internet and society more generally) and can be manipulated to enable unethical or criminal activity. For example, ChatGPT won’t give you instructions on how to hotwire a car, but if you say you need to hotwire a car to save a baby, the algorithm is happy to comply. Organizations that rely on generative AI models should reckon with reputational and legal risks involved in unintentionally publishing biased, offensive, or copyrighted content.
- Generative AI can have many benefits in healthcare, including
Improved patient care
Generative AI can help healthcare professionals make better decisions by analyzing medical data more efficiently.
Personalized medicine
Generative AI can create personalized treatment plans for patients based on their genetic information, medical history, and other factors.
More accurate diagnoses
Generative AI can improve medical imaging by generating high-resolution images from a large dataset of medical images.
Reduced human error
Generative AI can automate repetitive tasks like data entry and administrative processes, and correct mistakes in documentation. - Generative AI can have many benefits in healthcare, including
Streamlined workflows
Generative AI can optimize workflows by prioritizing tasks and allocating resources.
Clinical decision support
Generative AI can help with outcome prediction and clinical decision making.
Collaboration
Generative AI can allow specialists to collaborate in real-time, which can lead to better patient outcomes.
Democratization
Generative AI can allow patients to access their medical imaging records, which can help new specialists see their medical history and patients seek second opinions - Generative AI models can analyze vast patient data, including medical records, genetic information, and environmental factors. By integrating and analyzing these data points, AI models can identify patterns and relationships that may not be apparent to humans.
- Generative AI performs optimally in environments characterized by high repetition and low risk. This effectiveness stems from the technology’s reliance on historical data to identify patterns and make predictions, under the premise that future conditions will mirror those of the past. Utilizing such technology in low-risk situations, particularly where errors carry minor consequences, is prudent. This cautious approach offers several advantages: It enables health care providers and, more importantly, patients to gradually comprehend the AI’s capabilities and establish trust in its utility. Additionally, it affords AI developers valuable opportunities to rigorously test and refine their systems in a controlled environment before deployment in higher-stakes scenarios.
- Advanced AI models, especially those designed for medical analysis, prominently feature advanced techniques like convolutional neural networks (CNNs) and various deep learning frameworks. Here are some key aspects of the impact of generative AI in medical imaging:
Image synthesis: Generative models synthesize organ or tissue images, serving educational purposes like training medical professionals and simplifying medical condition explanations to patients through visually comprehensible representations.
Automated segmentation: Generative AI automates the segmentation of organs or abnormalities in medical images, efficiently saving time for healthcare professionals and streamlining the image analysis process.
Pathology prediction: Analyzing patterns in medical images, generative AI aids in predicting or identifying pathological conditions, facilitating early detection and intervention for improved patient outcomes. - Personalized medicine: Generative AI is crucial in advancing personalized medicine, which aims to provide tailored treatment plans based on individual patient data. Here is how generative AI is utilized:
Tailored treatment plans: Generative models can analyze patient data, including genetic information, medical history, and clinical data, to generate personalized treatment plans. This can aid in selecting the most effective therapies and predicting individual patient responses.
Predictive analytics for disease progression and treatment response: Generative AI can generate predictive models that estimate disease progression and treatment outcomes by analyzing large datasets and integrating various patient factors. This helps healthcare professionals make informed decisions regarding treatment strategies and optimize patient care.
Real-time clinical decision support: Gen AI provides clinicians with real-time, evidence-based recommendations for personalized treatment options based on a patient’s genetic profile. This accelerates decision-making by swiftly considering vast amounts of data with precision. - Ethical and legal compliance assistance: Gen AI aids in navigating ethical and legal considerations related to personalized medicine, ensuring adherence to privacy regulations and ethical standards. This builds patient trust and maintains compliance with healthcare laws.
Resource optimization in genetic testing: Gen AI contributes to resource optimization by streamlining workflows, automating routine tasks, and enhancing the efficiency of genetic testing processes. This is essential for overcoming resource limitations and making personalized medicine more accessible.
Pharmacogenomic optimization: Gen AI analyzes pharmacogenomic data to predict individual medication responses, enabling tailored drug prescriptions based on genetic factors. This optimizes treatment outcomes and minimizes adverse effects. 
- Medical research and data analysis
Generative AI techniques have immense potential in medical research and data analysis. Here are how generative AI aids in medical research and data analysis:
Data processing: Generative AI swiftly analyzes extensive medical data, automating data extraction and document reviews. This streamlines administrative processes, allowing researchers to focus more on critical aspects of their work.
Medical document summarization: Generative AI excels at summarizing lengthy medical documents, offering concise overviews for researchers. This accelerates comprehension and decision-making, especially when navigating extensive medical literature.
Trend identification and analysis: Processing large datasets, Generative AI identifies patterns and analyzes trends in medical research. This keeps researchers informed about the latest developments, fostering a proactive and informed approach in the field.
Optimizing resource utilization: Generative AI addresses resource constraints in medical research by automating tasks and optimizing available resources. This particularly benefits projects with limited funding or access to high-performance computing resources.
Predictive analytics insights: Leveraging historical medical data, Generative AI provides insights into potential outcomes, aiding researchers in making informed decisions and creating strategies for their medical research projects. - Large language models (LLMs) are a type of artificial intelligence (AI) program that can generate and recognize text. They are deep learning models that are trained on large amounts of text data, such as programming languages. LLMs are built on machine learning and use a type of neural network called a transformer model.
What’s the difference between a Large Language Model (LLM) and a General Pre-trained Transformer (GPT)?A large language model and a general pre-trained transformer both refer to advanced machine learning models based on the transformer architecture. However, they have some differences in their focus and application.
What’s the difference between a Large Language Model (LLM) and a General Pre-trained Transformer (GPT)?
The main difference between a large language model and a general pre-trained transformer is their focus and application. Large language models are specifically designed for natural language processing tasks, while general pre-trained transformers can be applied to a wider range of problems beyond language processing.- Large Language Model: A large language model, like OpenAI’s GPT (Generative Pre-trained Transformer) series, is specifically designed and trained for natural language processing tasks. These models are trained on vast amounts of text data and are capable of generating human-like text, understanding context, and answering questions. They can be fine-tuned for specific tasks like translation, summarization, or sentiment analysis. Examples of large language models include GPT-3, GPT-4, BERT, and RoBERTa.
- General Pre-trained Transformer: A general pre-trained transformer is a more broad term for models based on the transformer architecture. While these models can also be used for natural language processing tasks, they can be applied to a wider range of problems, including computer vision, speech recognition, and reinforcement learning. These models are pre-trained on large datasets and can be fine-tuned for specific tasks. Examples of general pre-trained transformers include ViT (Vision Transformer) for computer vision tasks and Conformer models for speech recognition tasks.
- Is OpenAI’s ChatGPT a Large Language Model?
Yes, OpenAI’s ChatGPT is a large language model. It is based on the GPT (Generative Pre-trained Transformer) architecture, which is specifically designed for natural language processing tasks. ChatGPT is trained on vast amounts of text data and is capable of generating human-like text, understanding context, and answering questions. It can be used for various applications, such as conversation, translation, summarization, and more.
ChatGPT is a fine-tuned version of the base GPT model to make it more suitable for generating conversational responses. Examples of GPT models include GPT-2, GPT-3, and GPT-4.
- IMPORTANCE Since the introduction of ChatGPT in late 2022, generative artificial intelligence (genAI) has elicited enormous enthusiasm and serious concerns.
OBSERVATIONS History has shown that general purpose technologies often fail to deliver their promised benefits for many years (“the productivity paradox of information technology”). Health care has several attributes that make the successful deployment of new technologies even more difficult than in other industries; these have challenged prior efforts to implement AI and electronic health records. However, genAI has unique properties that may shorten the usual lag between implementation and productivity and/or quality gains in health care. Moreover, the health care ecosystem has evolved to make it more receptive to genAI, and many health care organizations are poised to implement the complementary innovations in culture, leadership, workforce, and workflow often needed for digital innovations to flourish.
CONCLUSIONS AND RELEVANCE The ability of genAI to rapidly improve and the capacity of organizations to implement complementary innovations that allow IT tools to reach their potential are more advanced than in the past; thus, genAI is capable of delivering meaningful improvements in health care more rapidly than was the case with previous technologies.
Will Generative Artificial Intelligence Deliver on Its Promise in Health Care?
Robert M.Wachter, MD; Erik Brynjolfsson
JAMA. doi:10.1001/jama.2023.25054 - “While EHRs have cut the rate of medication errors and delivered numerous other benefits, the evidence that they have improved productivity is mixed, particularly when factoring in the HER associated increase in clinicians’ documentation burden The latest unanticipated consequence is the explosion in electronic messages coming from the patient portal to the physician’s HER inbox. Clinicians often cite the EHR as a key factor in their dissatisfaction with work and high levels of burnout.”
Will Generative Artificial Intelligence Deliver on Its Promise in Health Care?
Robert M.Wachter, MD; Erik Brynjolfsson
JAMA. doi:10.1001/jama.2023.25054 - “Finally, while “fail fast and iterate” is a reasonable mantra for a consumer-facing app, the stakes in health care are too high to tolerate flaws in the output of information technology (IT) tools that could result in patient harm. Moreover, if the use of an IT tool leads to a patient death, there is likely to be mainstream and social media attention, and potentially a malpractice case, to remind everyone of the risks.”
Will Generative Artificial Intelligence Deliver on Its Promise in Health Care?
Robert M.Wachter, MD; Erik Brynjolfsson
JAMA. doi:10.1001/jama.2023.25054 - “Early research on genAI outside of health care supports the premise that these tools have the capacity to deliver productivity and quality gains more quickly than prior technologies. One of us (E.B.) worked with colleagues to study the phased rollout of a genAI-based tool for assisting more than 5000 customer support agents in a software company.35 The agents given access to the tool had a 14% increase in productivity, accompanied by improvements in customer satisfaction and employee retention. Most of these improvements occurred within the first few months of genAI deployment and involved relatively small changes in the organization of the work. Interestingly, the least experienced and least skilled workers saw the biggest benefits, with productivity gains of 35%as they quickly ascended the learning curve.”
Will Generative Artificial Intelligence Deliver on Its Promise in Health Care?
Robert M.Wachter, MD; Erik Brynjolfsson
JAMA. doi:10.1001/jama.2023.25054 - “A recent analysis by economists at Harvard and the consulting firm McKinsey projected that the implementation of modern AI systems could lead to savings of 5% to 10% in health care spending (roughly$200-$360 billion per year in 2019dollars),mostly by addressing use cases in operations, corporate functions, and reimbursement. These savings may be an underestimate if genAI is ultimately successful in facilitating high value and evidence based care through effective clinical decision support.”
Will Generative Artificial Intelligence Deliver on Its Promise in Health Care?
Robert M.Wachter, MD; Erik Brynjolfsson
JAMA. doi:10.1001/jama.2023.25054 - “The productivity paradox of Its likely to rear its head with the implementation of genAI in medicine, just as it has with prior technologies, both inside and outside health care. In fact, when compared with other industries, health care has several attributes that increase the challenge of reaping the promised benefits of technology tools. Nevertheless, we are optimistic that the 2 key factors that have historically been critical in overcoming the productivity paradox— the ability of the digital tools to rapidly improve and the capacity of organizations to implement complementary innovations that allow IT tools to reach their potential—are more advanced than in the past. Because of this, we believe that genAI will deliver meaningful improvements in health care more rapidly than was the case with previous technologies.
Will Generative Artificial Intelligence Deliver on Its Promise in Health Care.
Robert M. Wachter, MD; Erik Brynjolfsson
JAMA. doi:10.1001/jama.2023.25054 - “But it does mean that what might have been a decades-long path for genAI to overcome the productivity paradox in health care may now be traversed in 5 to 10 years, and for some digitally advanced organizations, even sooner. None of this will happen automatically. GenAI developers will need to effectively address concerns regarding hallucinations, bias, safety, and affordability. Regulators will need to enact standards that facilitate trust in genAI without unduly stifling innovation. And, most important, health care leaders will need to put in place actionable roadmaps that prioritize the areas where genAI can create the greatest benefits for their organizations, paying close attention to those complementary innovations that remain necessary and striving to mitigate the known problems with genAI and any unanticipated consequences that emerge. Given the health care system’s outsized role in both human health and in economics, the stakes could hardly be higher.”
Will Generative Artificial Intelligence Deliver on Its Promise in Health Care?
Robert M.Wachter, MD; Erik Brynjolfsson
JAMA. doi:10.1001/jama.2023.25054
- About 4% of diagnostic interpretations contain clinically significant errors. Errors arise because many image interpretation tasks are not well suited for human capabilities. For example, finding a small nodule nestled among the pulmonary vessels is akin to a “needle in a haystack.” Accurately quantifying abnormalities, such as the size of an irregularly shaped tumor or the amount of calcium in the coronary arteries, can also challenge human capabilities. Correlating complex multimodal clinical data sources, such as radiology, genomics, and pathology, may be beyond human capabilities. But AI algorithms can readily perform these tasks. Thus, AI researchers will continue to develop these new capabilities as a complement to human perception.
The Future of AI and Informatics in Radiology: 10 Predictions
Curtis P. Langlotz
Radiology 2023; 309(1):e231114 - “Virtual assistants will bring these same efficiencies to radiologists who do not have the privilege of working with trainees. The combination of computer vision algorithms, which analyze images to identify findings, and large language models (LLMs), which are trained on massive data sets to generate text, will make this possible. Some computer vision algorithms can detect more than 70 findings in a single imaging study. Prompted by this list of findings, an LLM will draft a radiology report. Finally, the radiologist will edit and sign the report. The AI models could be periodically retrained from feedback obtained by comparing draft and final reports.”
The Future of AI and Informatics in Radiology: 10 Predictions
Curtis P. Langlotz
Radiology 2023; 309(1):e231114 - “Until the advent of modern machine learning methods in the past few years, it was unthinkable that some radiology studies would never be viewed by human eyes. But many electrocardiogram and Papanicolaou test interpretations have been human-free for years. Recent research predicts that workflows combining human and AI expertise can forgo human review of 63% of screening mammograms while increasing overall accuracy. Because screening is only a small part of radiologist work, these systems may slow the growth of the radiologist workforce but will not displace radiologists.”
The Future of AI and Informatics in Radiology: 10 Predictions
Curtis P. Langlotz
Radiology 2023; 309(1):e231114 - “But LLMs will soon be deployed for other radiology applications. Regulations against information blocking give patients ready access to their medical information. But the terminology radiologists use to communicate with requesting clinicians can mystify patients. The ability of LLMs to summarize information at an arbitrary reading level in the patient’s preferred language will help patients understand their reports. For example, here is the response of ChatGPT, an LLM developed by OpenAI, when asked to explain the circle of Willis at a fifth-grade reading level .”
The Future of AI and Informatics in Radiology: 10 Predictions
Curtis P. Langlotz
Radiology 2023; 309(1):e231114 
The Future of AI and Informatics in Radiology: 10 Predictions
Curtis P. Langlotz
Radiology 2023; 309(1):e231114- Over the past several years, the most accurate and generalizable AI systems have been trained on large diverse labeled data sets. Recent research suggests that pretraining on massive unlabeled data produces the most accurate systems. These systems, often called foundation models, can be fine-tuned on data from the deployment site to produce systems that are accurate for a wide range of tasks. These methods to optimize AI accuracy are on a collision course with medical software regulation. The U.S. FDA makes its decisions based on evidence from data about static products. The need to finetune foundational AI models to optimize their accuracy at a local site requires that products change after regulatory clearance. In the next decade, AI researchers, clinicians, ethicists, and regulators will devise flexible regulatory frameworks that allow monitoring and fine-tuning of algorithms on local data. The FDA’s proposed predetermined change control plans are a step in the right direction .
The Future of AI and Informatics in Radiology: 10 Predictions
Curtis P. Langlotz
Radiology 2023; 309(1):e231114 - “Academic institutions will continue to lead AI research and development because of their immediate access to all the necessary raw materials: massive stores of accessible clinical data, a workforce of students with deep technical knowledge, abundant high-performance computing, research teams with interdisciplinary expertise, close partnerships with industry, and relationships with health care delivery systems that serve as showcases and testbeds for their innovations .”
The Future of AI and Informatics in Radiology: 10 Predictions
Curtis P. Langlotz
Radiology 2023; 309(1):e231114
- ”Artificial intelligence (AI) tools used in medicine, like AI used in other fields, work by detecting patterns in large volumes of data. AI tools are able to detect these patterns because they can “learn,” or be trained to recognize, certain features in the data. However, medical AI tools trained with data that are skewed in some way can exhibit bias, and when that bias matches patterns of injustice, the use of the tools can lead to inequity and discrimination. Technical solutions such as attempting to fix biased clinical data used for AI training are well intentioned, but what undergirds all these initiatives is the notion that skewed clinical data are “garbage,” as in the computer science adage “garbage in, garbage out.” Instead, we propose thinking of clinical data as artifacts that, when examined, can be informative of societies and institutions in which they are found.”
Considering Biased Data as Informative Artifacts in AI-Assisted Health Care
Kadija Ferryman, Maxine Mackintosh, and Marzyeh Ghassemi
N Engl J Med 2023;389:833-8. - ”Viewing biased clinical data as artifacts can identify values, practices, and patterns of inequity in medicine and health care. Examining clinical data as artifacts can also provide alternatives to current methods of medical AI development. Moreover, this framing of data as artifacts expands the approach to fixing biased AI from a narrowly technical view to a sociotechnical perspective that considers historical and current social contexts as key factors in addressing bias. This broader approach contributes to the public health goal of understanding population inequities and also provides novel ways to use AI as a means of detecting patterns of racial and ethnic correction, missing data, and population inequities that are relevant to health equity. ”
Considering Biased Data as Informative Artifacts in AI-Assisted Health Care
Kadija Ferryman, Maxine Mackintosh, and Marzyeh Ghassemi
N Engl J Med 2023;389:833-8. - “The growing attention to bias within the AI and health care communities is a welcome development, especially as we continue to experience the ebbs and flows of the coronavirus disease 2019 pandemic. However, the harms of AI have often been imprecisely and narrowly considered as a data bias problem. Although there is value in innovating computational ways of altering data sets and engaging diverse participants in biomedical research, these cannot be the only solutions, and they should not rely on the implicit notion that past and current health data have little to offer AI research and development today.”
Considering Biased Data as Informative Artifacts in AI-Assisted Health Care
Kadija Ferryman, Maxine Mackintosh, and Marzyeh Ghassemi
N Engl J Med 2023;389:833-8. - ”Examining health care data as artifacts expands the technical approach to data bias in AI development, offering a sociotechnical approach that considers historical and current social contexts as important factors. This expanded approach serves the public health goal of understanding population inequities and suggests novel uses of AI to detect health equity–relevant data patterns. We propose this reframing so that the development of AI in health care can reflect our commitment and responsibility to ensure equitable health care now and in the future.”
Considering Biased Data as Informative Artifacts in AI-Assisted Health Care
Kadija Ferryman, Maxine Mackintosh, and Marzyeh Ghassemi
N Engl J Med 2023;389:833-8.
- “Our prespecified primary outcome was whether the model’s top diagnosis matched the final case diagnosis. Prespecified secondary outcomes were the presence of the final diagnosis in the model’s differential, differential length, and differential quality score using a previously published ordinal 5-point rating system based on accuracy and usefulness in which a score of 5 is given for a differential including the exact diagnosis and a score of 0 is given when no diagnoses are close).”
Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge
Zahir Kanjee et al
JAMA 2023 (in press) - “A generative AI model provided the correct diagnosis in its differential in 64% of challenging cases and as its top diagnosis in 39%. The finding compares favorably with existing differential diagnosis generators. A 2022 study evaluating the performance of 2 such models also using New England Journal of Medicine clinicopathological case conferences found that they identified the correct diagnosis in 58% to 68% of cases; the measure of quality was a simple dichotomy of useful vs not useful. GPT-4 provided a numerically superior mean differential quality score compared with an earlier version of one of these differential diagnosis generators (4.2 vs 3.8).”
Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge
Zahir Kanjee et al
JAMA 2023 (in press) - “Study limitations include some subjectivity in the outcome measure, which was mitigated with a standardized approach used in similar diagnostics literature. In some cases, important diagnostic information was not included in the AI prompt due to protocol limitations, likely leading to an underestimation of the model’s capabilities. Also, the agreement on the quality score between scorers was moderate. Generative AI is a promising adjunct to human cognition in diagnosis. The model evaluated in this study, similar to some other modern differential diagnosis generators, is a diagnostic “black box”; future research should investigate potential biases and diagnostic blind spots of generative AI models. Clinicopathologic conferences are best understood as diagnostic puzzles; once privacy and confidentiality concerns are addressed, studies should assess performance with data from real world patient encounters.”
Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge
Zahir Kanjee et al
JAMA 2023 (in press) - “Heterogeneous results with regard to the perceptions of the effects of AI on error occurrence, alert sensitivity and timely resources were reported. In contrast, fear of a loss of (professional) autonomy and difficulties in integrating AI into clinical workflows were unanimously reported to be hindering factors. On the other hand, training for the use of AI facilitated acceptance. Heterogeneous results may be explained by differences in the application and functioning of the different AI systems as well as inter-professional and interdisciplinary disparities.”
An integrative review on the acceptance of artificial intelligence among healthcare professionals in hospitals
Sophie Isabelle Lambert et al.
npj Digital Medicine (2023) 6:111 ; https://doi.org/10.1038/s41746-023-00852-5 - “To conclude, in order to facilitate acceptance of AI among healthcare professionals it is advisable to integrate end-users in the early stages of AI development as well as to offer needs-adjusted training for the use of AI in healthcare and providing adequate infrastructure.”
An integrative review on the acceptance of artificial intelligence among healthcare professionals in hospitals
Sophie Isabelle Lambert et al.
npj Digital Medicine (2023) 6:111 ; https://doi.org/10.1038/s41746-023-00852-5 - AI developers are trying to apply their technologies in many fields such as engineering, gaming and education1. Lately, the development of AI technologies has expanded to medical practice and its implementation in complex healthcare work environments has begun, Choudhury et al.have defined AI in healthcare as ‘an adaptive technology leveraging advanced statistical algorithm(s) to analyze structured and unstructured medical data, often retrospectively, with the final goal of predicting a future outcome, identifying hidden patterns, and extracting actionable information with clinical and situational relevance’ .
An integrative review on the acceptance of artificial intelligence among healthcare professionals in hospitals
Sophie Isabelle Lambert et al.
npj Digital Medicine (2023) 6:111 ; https://doi.org/10.1038/s41746-023-00852-5 - “This integrative review aims to unravel the variety of reported causes for the limited acceptance as well as facilitating factors for the acceptance of AI usage in the hospital setting to date. The assessment and analysis of reasons for distrust and limited usage are of utmost importance to face the increasing demands and challenges of the healthcare system as well as for the development of adequate, needs-driven AI systems while acknowledging their associated limitations. This includes the identification of factors influencing the acceptance of AI as well as a discussion of the mechanisms associated with the acceptance of AI in light of current literature. This review’s findings aim to serve as a basis for further practical recommendations to improve healthcare workers’ acceptance of AI in the hospital setting and thereby harness the full potential of AI.”
An integrative review on the acceptance of artificial intelligence among healthcare professionals in hospitals
Sophie Isabelle Lambert et al.
npj Digital Medicine (2023) 6:111 ; https://doi.org/10.1038/s41746-023-00852-5