A Framework for Cardboard based Augmented Reality

Abstract

We propose a generic framework for immersive augmented reality experiences based on a low-cost head mounted viewer (HMV) such as Google Cardboard. The proposed framework provides a functional implementation structure that can act as the baseline for a large variety of applications. It consists of a camera for capturing the real world image, a module for frame sorting, which provides a frame by frame output for processing and the provision for input from di erent on-board as well as external sensors. A computer vision module processes the incoming video to generate meaningful information that is used together with the input from di erent sensors by the main processing module to take decisions regarding position and pose of virtual objects with respect to the real scene. The module for placement and positioning of 3D virtual objects performs transformations on the virtual objects such that they are consistently overlaid on the real world image and rendered on the display screen. This module is responsible for ensuring that the nal image rendered is consistent across di erent head mounted viewers. Furthermore, di erent modalities of interaction are incorporated including direct voice input and using an external controller. Communication with external sensors and other neighboring systems is also supported. As a proof-of-concept a head mounted viewer application has been implemented that provides navigation for drivers in NUST H-12 Campus. A user can select their destination and the application guides the user to their destination through the head mounted viewer display. Using the on-board sensors, the user is routed via the shortest vehicular path by displaying virtual directional arrows superimposed on the real view.