Design of a Spatially Anchoring Framework Using an Augmented Reality Head-Mounted Device and Fisheye Lens

The development of improved Augmented Reality (AR) Head-Mounted Devices (HMDs) have led to many potential applications for augmented reality. In the case of surgery, an HMD could be used as an assistive tool to address some deficits that popular neuronavigation devices have, such as a disturbance to surgical workflow and surgeon comfort.

This thesis aims to bring an HMD-based overlay framework that can be used in the operating room. Through a combination of Android Studio, OpenCV, and OpenGL, a method to localize an HMD (ODG R9s) was created. This framework used a fisheye lens and a marker-based tracking method to localize the headset and renders a spatially-anchored 3D model in real time. A “Focus” mode was also developed in stabilizing the camera pose estimation and translational and rotational errors were quantified.