Microsoft Hololens, Augmented Reality as part of the Mixed Reality experience

Picture showing Microsoft HoloLens, used with permission from Microsoft

Augmented Reality or Mixed Reality?

Definition

Augmented Reality is a live direct or indirect view of a physical, real-world environment whose elements are “augmented” by computer-generated or extracted real-world sensory input such as sound, video, graphics or GPS data (Wikipedia). Therefore it’s a section of Mixed Reality (Image 1 “Reality-Virtuality-Continuum”).

A common example for an application of Augmented Reality would be the insertion of further information during the live broadcast of a soccer game. Offside lines or goal distances during a free kick are digitally shown on TV. The augmentation of every reality impression, together with all extra information, occurs live (in real time) and is able to respond to every human sense modality. However, the term “augmented reality” is often used in connection with a visually extended depiction and therefore functionally often reduced to extensions of only the visual perception of its users.

In everyday life, one’s auditory sense can also benefit of AR-functions. Voice announcements in public transports are most often automatized and computer-controlled. The software tracks down the recent location of the tramcar and based on this information, determines the next stop. As a result the passenger can hear a combination of different pieces of text and hence knows about the next stop and direction of the exit.

 

Requirements

In the following, typical requirements for a mobile AR solution based on image markers are listed.

Hardware requirements:

  • Sensors to capture real life surroundings and orient themselves in space (e.g. RGB Camera, GPS, Gyroscope)
  • Interfaces to register user inputs (e.g. Touchscreen, Microphone)
  • Computing power/ Processors to process and calculate data
  • Interfaces for the output of data and information (e.g. Display, Earphones, Vibration)


Software
requirements:
Technology for the purpose of …

  • Analysing camera pictures (Image analysis)
  • Analysing different sensor data (GPS, Acceleration Sensor, Gyrosope)
  • Linking existing data sources and resources (e.g. Database Interfaces etc.)
  • Managing user inputs and sequence control systems (z.B. Game-Engine)
  • Superimposing the camera input with virtual augmentations (Image Processing)

 

History

AR can be traced back to the 1960´s. Mobile Augmented Reality however, is relatively new, yet developing very quickly. The first AR-Browser-Apps named junaio, Layar and Wikitude have been developed between 2008 and 2010. With the appearance of the iPhone 3Gs, which had the necessary sensors and more computing power, mobile AR became interesting for mass market in 2009. Over the course of time, companies such as metaio, Total Immersion and Qualcomm established themselves as Software-Providers for Augmented Reality SDKs, which have been provided on a license model basis  to other developers. Due to the growth of these companies and software solutions respectively, other technology groups took them over. Therefore in October 2015, PTC bought QUALCOMM’s vuforia SDK for 65 mil USD. Shortly before that (05-2015), Apple acquired the company metaio and their technology portfolio. Almost contemporaneously, Microsoft announced new data headset, the HoloLenses. Forerunners to those are, among others, the data goggles by the company Vuzix and the Google Glass. Since mid-2015 two basic progresses in the technology sector for AR solutions have been developed. For one thing the development of already existing approaches of solutions for mobile “handheld” devices, such as Smartphone, Tablet and Phablet (mostly changes in software e.g. Apples ARKit or Googles ARCore, both becoming native AR-technologies of iOS and Android respectively) and for another the investigation and production of new AR-Output Devices (mostly smartglasses, data goggles or camera and sensor technology). Not only Microsoft HoloLens is trying to achieve those developments but also other companies, that run the projects “Meta” and “Magic Leap”.

Reality-Virtuality-Continuum

The R-V-Continuum (or Immersive Computing Spectrum) pictures the continuous transition of real environment (on the left) up to virtual environment (on the right) and gives every different section appropriate terms. Mixed Reality is the part of the R-V-Continuum that shows the range of the integral real and the whole virtual environment. Mixed Reality therefore comprises every section in which reality is mixed with virtuality (or vice versa). Augmented Reality is consequently part of Mixed Reality.

Reality-Virtuality-Continuum: From reality over augmented, mixed virtual reality to virtuality

Augmented Reality Applications and Examples

Where do Augmented Reality and Mixed Reality Solutions appear in every day life? How can Mixed Reality Headsets like the Microsoft HoloLens revolutionize the Human-Computer Interaction (HCI) in private but also professional environments in the future? In the following, some examples of AR/MR usecases and their advantages for various industries and business units will be pointed out.

Real Estate / Architecture / Interior Design

Augmented Reality architecture

Project for KWAG – AR App Leopold Carré – 2015

  • High Information rate due to interactive, animated depiction of planned or existing Real Estate as AR-Miniature Model
  • Intuitive access to the context of the building (neighbourhood development, existing objects, lighting and shadows, prominent properties of the object) due to the processing of information in the AR-Scene
  • Convincing presentation of construction project via pre-visualization of the buildings and their construction phases in life size in the foundation ditch
  • AR-Applications as tools for interactive planning of the interior design help the client’s imagination
  • and others…

Fairs / Exhibitions

Augmented Reality at trade shows, exhibitions and fairs

Project for OWA – AR App for exhibition booth – 2017

  • Maximum benefit from the exhibition area via virtual presentation of products while being shown in real size at the same time
  • Depiction of additional information on exhibits (Text, Images, Moving Images, 3D Animation) with linear or non-linear user guides via the AR-App
  • Cost savings due to reduced transport costs for exhibitors (products can be transported as virtual exhibits)
  • Uncomplicated clients approach via Augmented Reality trade-fair concepts (crowd-puller, Icebreaker, innovation feature) with smart devices or as kiosk system (digital signage)
  • and others…