Microsoft Hololens, Augmented Reality as part of the Mixed Reality experience

Picture showing Microsoft HoloLens, used with permission from Microsoft

Augmented Reality or Mixed Reality?

Definition

Augmented Reality is a live direct or indirect view of a physical, real-world environment whose elements are “augmented” by computer-generated or extracted real-world sensory input such as sound, video, graphics or GPS data (Wikipedia). Therefore it’s a section of Mixed Reality (Image 1 “Reality-Virtuality-Continuum”).

A common example for an application of Augmented Reality would be the insertion of further information during the live broadcast of a soccer game. Offside lines or goal distances during a free kick are digitally shown on TV. The augmentation of every reality impression, together with all extra information, occurs live (in real time) and is able to respond to every human sense modality. However, the term “augmented reality” is often used in connection with a visually extended depiction and therefore functionally often reduced to extensions of only the visual perception of its users.

In everyday life, one’s auditory sense can also benefit of AR-functions. Voice announcements in public transports are most often automatized and computer-controlled. The software tracks down the recent location of the tramcar and based on this information, determines the next stop. As a result the passenger can hear a combination of different pieces of text and hence knows about the next stop and direction of the exit.

 

Requirements

In the following, typical requirements for a mobile AR solution based on image markers are listed.

Hardware requirements:

  • Sensors to capture real life surroundings and orient themselves in space (e.g. RGB Camera, GPS, Gyroscope)
  • Interfaces to register user inputs (e.g. Touchscreen, Microphone)
  • Computing power/ Processors to process and calculate data
  • Interfaces for the output of data and information (e.g. Display, Earphones, Vibration)


Software
requirements:
Technology for the purpose of …

  • Analysing camera pictures (Image analysis)
  • Analysing different sensor data (GPS, Acceleration Sensor, Gyrosope)
  • Linking existing data sources and resources (e.g. Database Interfaces etc.)
  • Managing user inputs and sequence control systems (z.B. Game-Engine)
  • Superimposing the camera input with virtual augmentations (Image Processing)

 

History

AR can be traced back to the 1960´s. Mobile Augmented Reality however, is relatively new, yet developing very quickly. The first AR-Browser-Apps named junaio, Layar and Wikitude have been developed between 2008 and 2010. With the appearance of the iPhone 3Gs, which had the necessary sensors and more computing power, mobile AR became interesting for mass market in 2009. Over the course of time, companies such as metaio, Total Immersion and Qualcomm established themselves as Software-Providers for Augmented Reality SDKs, which have been provided on a license model basis  to other developers. Due to the growth of these companies and software solutions respectively, other technology groups took them over. Therefore in October 2015, PTC bought QUALCOMM’s vuforia SDK for 65 mil USD. Shortly before that (05-2015), Apple acquired the company metaio and their technology portfolio. Almost contemporaneously, Microsoft announced new data headset, the HoloLenses. Forerunners to those are, among others, the data goggles by the company Vuzix and the Google Glass. Since mid-2015 two basic progresses in the technology sector for AR solutions have been developed. For one thing the development of already existing approaches of solutions for mobile “handheld” devices, such as Smartphone, Tablet and Phablet (mostly changes in software e.g. Apples ARKit or Googles ARCore, both becoming native AR-technologies of iOS and Android respectively) and for another the investigation and production of new AR-Output Devices (mostly smartglasses, data goggles or camera and sensor technology). Not only Microsoft HoloLens is trying to achieve those developments but also other companies, that run the projects “Meta” and “Magic Leap”.

Reality-Virtuality-Continuum

The R-V-Continuum (or Immersive Computing Spectrum) pictures the continuous transition of real environment (on the left) up to virtual environment (on the right) and gives every different section appropriate terms. Mixed Reality is the part of the R-V-Continuum that shows the range of the integral real and the whole virtual environment. Mixed Reality therefore comprises every section in which reality is mixed with virtuality (or vice versa). Augmented Reality is consequently part of Mixed Reality.