3D Imaging in Radiology: What A Long Strange Trip It's Been3D Imaging in Radiology: What A Long Strange Trip It's Been Elliot K. Fishman MD FACR Department of Radiology Johns Hopkins Hospital |
“What a Long Strange Trip It’s Been” is a term often referring to the band the Grateful Dead but also is probably a good description of the road that 3D imaging has taken in Radiology. We have been part of this journey since the mid 1980’s and this exhibit describes some of the changes we have seen and looks forward to where the future might be.  |
3D Imaging has and continues to be dependent on technology and the technologic advances in computer hardware and software. The hardware costs for state of the art 3D imaging in the 1980’s was $250-350,000 and now is usually under $10,000. Software development has matched the hardware advances and networking has allowed for the seamless and secure transfer of information. The other critical advance has been the CT scanners themselves where we have gone from datasets of under 100 slices to 3-5,000 slices. Scan slice thickness has gone from 3-4 mm to under 1mm. The introduction of Dual Source CT has also provided new capabilities for creating optimal datasets.  |
What has not changed over the years perhaps is the “big picture” role of 3D imaging? When we consider its role in medical imaging we typically like to divide things into specific channels.
|
3D medical imaging has always been closely aligned with 3D imaging or visualization in other domains. Whether the developments were in the world of movies (Lucasfilms, Pixar, Disney), video games (Microsoft, NVIDIA) or technology (Department of Defense, NASA) we shared a common need for "visualization" of ever increasing size and complex datasets. Two excellent quotes on visualization are provided and fit perfectly in the medical domain. |
“ Scientific Visualization is concerned with exploring data and information in such a way as to gain understanding and insight into the data. The goal of scientific visualization is to promote a deeper level of understanding of the data under investigation and to foster new insight into the underlying processes, relying on the humans' powerful ability to visualize.” Scientific Visualization, Techniques and Applications, R.A. Earnshaw (1992) “Visualization is the process of transforming information into a visual form, enabling users to observe the information. The resulting visual display enables the scientist or engineer to perceive visually features which are hidden in the data but nevertheless are needed for data exploration and analysis.” N. Gershon, From Perception to Visualization (1994) |
1985-2017: What has changed?
|
3D Medical Imaging: The Changes Than Changed the Landscape
|
3D Medical Imaging Hardware: Hopkins CT Experience
|
3D Medical Imaging Hardware: Hopkins CT Experience
|
The Images tell the story of the journey. The 3D software did the processing on a DEC PDP-11 Computer which was part of the room full of computers for the CT scanner.  |
The Pixar Image Computer was introduced in 1985-1986 and was the best performing system of its time doing an amazing 40 MIPS. The combination of Pixar Hardware and Software (VRT) began the era of great interest in CT post processing.  |
LucasFilms 1985 (Pixar Image Computer). The original images which seem crude by current standards showed the use of both RGB and an Alpha channel and we could now have images that showed bone, muscle and soft tissue.  |
3D imaging from Hopkins begins at 4:57 with other potential applications for the Pixar Image Computer also shown. This was from 1987. The CT reconstruction of 50-60 CT slices took over 24 hours to create 84 images that were then visualized as a video. |
The introduction of the Pixar technology became front page news in many of the initial sites that were trying to develop medical applications including Johns Hopkins Hospital.  |
“ Volumetric rendering differs from surface rendering in that all the information from the CT scans is preserved, not just surface boundaries. Object thickness and internal contours can be seen in the 3D projection.” Volumetric Rendering Technique: Applications for Three-dimensional Imaging of the Hip Fishman EK, Drebin RA, Ney DR et al. Radiology 1987 Jun;163(3):737-738  Volume Rendering was considered one of the key discoveries in the history of CT by Diagnosting Imaging in this timeline. The team at Hopkins was honored to share the credit with the Pixar team. |
Image quality and speed on the Pixar computer produced some impressive images considering the datasets were 4 mm thick and reconstructed with 3 mm overlap. The 3D imaging began to play a critical role in orthopedic surgery.  |
Here are some of the our original published work trying to document the role for 3D imaging in clinical practice. Some of this early work was crucial in getting reimbursement for 3D studies in practice.
|
Here are some of the our original published work trying to document the role for 3D imaging in clinical practice. Some of this early work was crucial in getting reimbursement for 3D studies in practice.
|
Three-dimensional reconstruction of the Human Body Fishman, EK, Drebin RA, Hruban RH et al. AJR 1988 Jun;150(6):1419-1420  |
Over time Pixar left the computer hardware and medical business to concentrate on the movies (Toy Story) and led by Ed Catmull and Steve Jobs found great success and became part of Disney in the late 1990’s. Many other companies pursued the hardware for image processing with Silicon Graphics (SGI) being the most famous company for nearly a decade. We used many different platforms and it is amazing how we have gone from large computers like the Onyx (SGI) to the O2 (SGI) to the NeXT Computer and eventully to special purpose boards (Nvidia) with the interactive display now possible on iPads and iPhones.  Silicon Graphics ONYX, and O2 and the NeXT Computer |
"Server side rendering and processing allows manipulation of images to be done where the information is stored. The image current being viewed on the devise is the only one that is stored at that point in time. Because there is minimal information on the devise at any given time, the method of viewing radiology images is the most secure." Cybersecurity in Radiology: Access of Public Hot Spots and Public Wi-Fi and Prevention of Cybercrimes and HIPAA Violations Gerard P et al. AJR 2013; 201:1186-1189 |
Review of 3D data on mobile devices can provide the ability to review images with patients at bedside or the clinic and well as provide low cost mobile solutions for radiologists and non-radiologists alike. Currently applications like Webviewer (Siemens Medical) are available in the Apple store and run on the iPad and iPhone. Numerous vendors provide similar products although widespread adoption has been slow.  |
We have long predicted that 3D imaging would become mainstream in CT scanning (see article quotes). We still believe it will someday.  |
3D Imaging has played a major role across imaging and its clinical applications continues to grow. Some of the applications include;
|
Volume Rendering Technique (VRT)-Facts
|
The limitations of MIP vs VRT are illustrated in this potential renal donor where the location of the renal vein relative to the aorta can be a source of confusion  |
The information gained from going from axial CT to Multiplanar CT (MPR) to MIP and VRT is nicely shown in this case of active Crohns Disease.  |
Do you see the duodenal mass which is a carcinoid tumor? The 3D VRT in coronal plane nicely defines the 1 cm vascular lesion which was missed on the axial CT scan.  |
Angiodysplasia as a source of GI Bleed recognized on the 3D display  |
Interactive Analysis of 3D datasets is ideally done in real time by the radiologist combining axial, MPR and 3D imaging. FinalDiagnosis is FNH (Focal Nodular Hyperplasia)  |
"Although these routine CT examinations may be diagnostic for pancreatic adenocarcinoma, they are inadequate for disease extent assessment given the lack of optimal multiphasic enhancement and use of thicker slice selection. These factors limit the ability to generate highly quality reformatted images and 3D reconstructions that are often necessary for accurate staging. It is therefore essential that these patients undergo MDCT angiogram using a dedicated pancreatic protocol." Pancreatic Ductal Adenocarcinoma Radiology Reporting Template: Consensus Statement of the Society of Abdominal Radiology (SAR), and the American Pancreatic Association (APA) Al-Hawary MH, Francis IR, Chari ST, Fishman EK et al. Radiology 2014;270:248-260 |
Interactive Volume Rendering of the Pancreas and Arterial Map  |
Main Duct IPMN with VRT Interactive 3D analysis |
Serous Cystadenoma (MIP vs VRT) Interactive 3D analysis |
Conclusions "The present study showed that VRT and axial images produce almost identical values of CI, with the advantages of greater ease of execution and a time saving of almost 50% for 3D VRT images. In addition, VRT provides an integrated perspective that can better assist surgeons in clinical decision making and in operative planning, suggesting this technique as a possible standard method for CI measurement." Value of three-dimensional volume rendering images in the assessment of the centrality index for preoperative planning in patients with renal masses. Sofia C et al. Clin Radiol 2017 Jan;72(1):33-40 |
Clear Cell RCC  |
"The genetic makeup of clear cell RCCs (ccRCCs) affects their imaging features at multidetector CT examinations. Multidetector CT imaging characteristics may help suggest differences at the cytogenetic level among ccRCCs." Clear Cell Renal Cell Carcinoma: Multiphasic Multidetector CT Imaging Features Help Predict Genetic Karyotypes Sauk SC et al. Radiology 2011; 261:854-862 |
New rendering algorithms will help push 3D rendering forward. A novel technique called Cinematic Rendering seems to have some advantages over classic volume renderings.  |
“In addition to the fact that photo-realistic volume renderings tend to be aesthetically more pleasing, it has been shown that realistic lighting contributes to 3D understanding and can improve depth-related task performance . With this work and the implementation that we have made available, we hope to contribute to the uptake of realistic illumination in interactive direct volume rendering applications.” Exposure Render: An Interactive Photo-Realistic Volume Rendering Framework Thomas Kroes et al. PLOS ONE 7(7): e38586. doi: 10.1371/journal.pone.0038586 |
“Cinematic rendering produces volume rendered images with photorealistic image quality. It uses a global illumination model, which takes direct and indirect illumination into account when constructing an image, to achieve rendering quality. The mathematical models that describe this visualization technology include complex integral equations that are solved numerically using the Monte Carlo integration. The result of the integration is a numerical rendering algorithm known as path tracing: thousands of light rays are traced to compute the resulting image.” MDCT angiography with 3D rendering: A novel cinematic rendering algorithm for enhanced anatomic detail Johnson PT, Schneider R, Lugo-Fagundo C, Johnson M, Fishman EK AJR Am J Roentgenol. 2017 Aug;209(2):309-312 |
Cinematic Rendering of the Normal Pancreas  |
1 cm PNET of the Pancreas  |
Trauma is a classic clinical application for 3D. Here is a mandibular fracture imaged with cinematic rendering  |
Visualization of the neck with detailed vascular, soft tissue and muscle anatomy is often critical across a range of applications from trauma to infection to oncology  |
3D Cinematic Rendering Can Provide Detail of the Vessels as well as Muscle and the soft tissue.  |
New techniques in 3D can be used beyond the walls of the hospital  |
3D imaging can be used as the basis for printing complex anatomy with 3D printing  |
3D mapping solves problems that may even be centuries old  |
Summary The journey we are on changes every day and we look forward to new opportunities for improving the role of 3D in medical imaging. With a limited number of slides we could barely scratch the surface of the field and the changes over the years. We are optimistic that the best is yet to come. "You can't connect the dots looking forward; you can only connect them looking backward. So you have to trust that the dots will somehow connect in your future. You have to trust in something - your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life." Steve Jobs |