Tele-Immersion for Advanced Surgical Training and for Assistance During Surgical Procedures

Henry Fuchs

Department of Computer Science

University of North Carolina

Abstract:

Recent results from projects in tele-immersion and in augmented visualization for breast biopsies have inspired a new project with far more ambitious goals: improving advanced surgical training by immersive tele-presence within pre-recorded trauma cases. In this collaboration between UNC, Brown University, University of Pennsylvania, and the Pittsburgh Supercomputing Center, we are building a system for recording the events in a surgical theater with 50-100 video cameras. After the recordings, an instructor should be able to edit and enhance the immersive 3D recordings with hypermedia aids for later class demonstrations and independent immersive student study. We are developing new techniques for continuously recording the activities in the surgical theater, for saving indicated periods of interest, for reconstructing 3D time sequences (with some regions in great detail), for rendering these detailed reconstructions in real-time, and for presenting the stereo images using head-mounted displays and projectors. Although portions of these capabilities have already been demonstrated by our group and others, performing all the steps simultaneously (from acquisition to presentation to interaction) poses unprecedented technical challenges. If successful, however, such tele-immersive systems should enable a wide variety of training, consultation and collaboration activities, including a particular interest of ours, real-time surgical consultation over long distances.

Brief Biography:

Henry Fuchs (PhD, Utah, 1975) is Federico Gil Professor of computer science, adjunct professor of biomedical engineering and adjunct professor of radiation oncology at UNC Chapel Hill. He has co-authored over 100 papers in computer graphics, virtual environments, and medical applications. He has served on the (USA) National Research Council’s Computer Science and Telecommunications Board, on evaluation committees and study groups for NSF, NIH, and DARPA, and on several industrial advisory boards. He has been a member of the editorial boards of ACM Transactions on Graphics, IEEE Transactions on Visualization and Computer Graphics, and the Virtual Reality Society Journal. He received the 1992 SIGGRAPH Computer Graphics Achievement Award, the 1992 Academic Award of the National Computer Graphics Association, and the 1997 Satava Award of the Medicine Meets Virtual Reality Conference. Fuchs is a member of the National Academy of Engineering and a fellow of the American Academy of Arts and Sciences.