Abstract:
We propose a generic framework for immersive augmented reality experiences
based on a low-cost head mounted viewer (HMV) such as Google Cardboard. The
proposed framework provides a functional implementation structure that can act as
the baseline for a large variety of applications. It consists of a camera for capturing
the real world image, a module for frame sorting, which provides a frame by frame
output for processing and the provision for input from di erent on-board as well
as external sensors. A computer vision module processes the incoming video to
generate meaningful information that is used together with the input from di erent
sensors by the main processing module to take decisions regarding position and
pose of virtual objects with respect to the real scene. The module for placement
and positioning of 3D virtual objects performs transformations on the virtual objects
such that they are consistently overlaid on the real world image and rendered on
the display screen. This module is responsible for ensuring that the nal image
rendered is consistent across di erent head mounted viewers. Furthermore, di erent
modalities of interaction are incorporated including direct voice input and using an
external controller. Communication with external sensors and other neighboring
systems is also supported. As a proof-of-concept a head mounted viewer application
has been implemented that provides navigation for drivers in NUST H-12 Campus.
A user can select their destination and the application guides the user to their
destination through the head mounted viewer display. Using the on-board sensors,
the user is routed via the shortest vehicular path by displaying virtual directional
arrows superimposed on the real view.
