Augmented Reality App and AR Browser

AR Technologies can be used in various industries and business sectors. Due to the variety of different types of application and external conditions, two typical approaches for the development of AR-Apps have been established. For one thing, custom made AR Apps can be produced with complete configurability of every feature in order to achieve ideal solutions for every challenge. For another, already existing AR Browser Apps, that are seen as some sort of publishing platform and offer standardized features (sometimes even as an Editing-Tool). Even though AR-Browser Apps permit their producers a comparatively easy access into the AR-Content-Progressing and Publishing, due to extensive automation and strictly defined templates of features, the possibilities of using more complex AR-Functions however is mostly impossible and own contents can only be used restrictively to be sales efficient and advertised (less branding possibilities). Custom made AR-Apps however can be realized in a very complex manner, accomplish even difficult technological challenges and can be advertised and published as an independent application. In the following, both approaches will be explained.

Augmented Reality Apps

What is an Augmented Reality App?

AR Applications offer their users additional information on their location, objects (e.g. products) in their immediate environment or enable the insertion of virtual objects on the display respresenting their real environment. In doing so, AR Apps can be used in almost every industry and can also be beneficial for entertainment, training, improvement of productivity, communication, sales increase. Augmented Reality can appear as various display formats, such as simple insertion of texts, videos and audio data, 360° Panorama images, 360° turn around images, animated 2D and 3D contents and also linear and interactive mechanisms for user instructions and interactions. The AR App can be used as simple client applications on the terminal device or more complex client-server applications on linked systems. As some properties of AR solutions are highly dependent on the purposse of use, AR Apps are usually made-to-measure, individual products based on Augmented Reality SDKs (Software-Development-Kits, etc.) and other basic technologies (e.g. Apple ARKit, Google ARCore, vuforia, etc.).

Which AR Apps exist?

Augmented Reality Apps exist in various industries and divisions. Due to the variety of the existing apps and the fact that they are published on various platform (Smartphone, Tablet, Computer, Headset, etc.) and operation systems, it is hardly possible to show a complete list of AR-Apps. In the following we have listed some of the best-known examples. Furthermore, more detail and information in the category “What is Augmented Reality?” is available (the difference between AR, MR and VR will also be explained).

• LEGO Digital Box
  Running in a Kiosk-System at the Point of Sale (LEGO-Stores) to communicate the product and support sales
• IKEA App
  Running as a smartphone App: addition to the IKEA catalogue (AR-Print/Interactive-Print) to preview the products
• Pokémon GO
  Running a s a smartphone App: most famous representative for AR-Games

Hint: The mentioned examples are AR-Apps developed by other companies, not ours.

How do Augmented Reality Apps work?

In order to realize reality visually and expand it due to its context, the app needs to “understand” the situation or environment in which it is being used. Various technologies (accessable as AR SDKs) are used for this – most of them are basic technologies using 2D Image marker recognition. More complex SDKs even offer 3D object recognition, SLAM or Visual Inertial Odometry (VIO) processes and might even allow GEO-Positioning via GPS. Most of these approaches analyse the picture of the final device and add audio-visual contents (e.g. Texts, Pictures, Videos, even 2D or 3D animations, etc.). The expanded reality is then showed on the display. As the complete process is happening simultaneously, the user thinks that the virtual additions are right on their screen and therefore a part (if even only on screen) of their reality.

Overview of the details of different Tracking-Approaches:

2D Image marker / Tracking marker

The term Image marker or also tracking marker usually describes an Image, which is usually stored in an AR-App or the related Cloud Service for comparison purposes, and therefore already known. The comparison is usually made via the analysis of light and dark in the picture and provides so called “Image-Features” in the form of contrast points. During the process of an application, the active camera picture is analysed and everything visible on the video picture is checked and compared on the Image markers (and their contrast point pattern). If this analysis matches, additional information can be assigned to the identified image marker and in addition it is possible match the position and rotation of virtual contents with the image marker based on the camera’s perspective. Therefore augmentations/holograms can be shown accordingly. Image markers are so to say a little bit more than QR-Codes. They do not only refer to additional data but also help to initialize a 3D coordination system in order to super impose the camera’s view.

3D Objekt recognition

Not only can 2D markers be replaced with the 3D object recognition, but also more complex applications can be realised. Instead of storing a 2D image reference, a 3D volume reference in the form of a Point Cloud is stored in the system. A camera repeatedly films the real object, analyses the video image and compares it to the 3D reference. To simplify this: the image analysis tries to copy the contrast points and edges of the video image and use them as a comparison to the stored 3D data basis. If this complex analysis identifies a known object in the camera picture, additional and linked information (text, video, 2D and 3D graphics, etc.) can be augmented on the display correctly in the perspective of the camera’s view. A huge advantage compared to the 2D image marker variation is, that the real object can be seen from all sides and steadily augmented.

SLAM (Simultaneous Localization And Mapping)

Solutions for SLAM (simultaneous location and mapping) features usually involve methods for constructing or updating a map of an unknown environment while simultaneously keeping track of the smartphone’s location within it. Initially, SLAM algorithms have been used in the context of autonomous robots. However they are also of high importance for AR and MR scenarios. AR and MR applications usually develop their full potential when the user’s device and app know as much as possible about the real environment and therefore the context. With the help of SLAM AR and MR solutions have the possibility to find their ways around in unknown territory and therefore collect information on environment and context of the scenery – an advantage that makes new ways of application possible, even without the use of reference markers.

Visual Inertial Odometry (VIO)

Processes that use the data of cameras (pictures), position and acceleration (Gyroscope and acceleration sensors) to identify the position, orientation and movement of a device in space are called “Visual Inertial Odometry”. Further usage of data leads to similar results as in the SLAM process and in addition offers a very precise comparison of lengths, sizes and distances. Therefore, virtual objects can be shown in realistic size dimensions without standardized tracking markers.

GEO-Positioning / Location Based Service (LBS)

As a “Location Based Service”, AR Apps show relevant information on the current location of its user. In order to do this, the users position is analysed via GPS and stored content will be loaded in its defined environment. Next to details on close buildings, shops and sights, virtual objects can be shown as virtual objects as 3D models or interactive elements can be implemented. A well-known example for an intensive use would be the App Pokémon GO, where the device/app tracks the users location, shows it as part of a map and adds virtual elements (Pokémon, Arenas, etc.). The users position and the position of a virtual addition is analysed and transferred. This creates an illusion of Pokémon around everyone’s neighbourhood.

What is an Augmented Reality SDK?

AR SDKs offer advanced technologies and tools to create AR solutions for the AR App developers. They contain important fundamental functions, such as image or object recognition, SLAM and different other software algorithms and partly render engines to show the content. Best-known SDKs would be the ARKit (Apple – after metaio’s Akquise), ARCore (Google), vuforia (PTC – used to be Qualcomm’s QCAR), Wikitude, MaxST, Kudan and ARToolKit, whereas the quality and types of software packages differs in their AR Technology elements.

	ARKit	ARCore	Wikitude	vuforia	Kudan
Operating system	iOS	Android	iOS / Android	iOS / Android / Windows UWP	iOS / Android
Maximum distance to marker (detection / tracking)	-	-	0.8m / 3m	1.2m / 3.7m	0.8m / 3m
Tracking quality for static marker	8/10	2/10	10/10	10/10	9/10
Tracking quality for moving marker	6/10	2/10	8/10	6/10	6/10
2D Tracking	Yes	Yes	Yes	Yes	Yes
3D Tracking	No	No	Yes	Yes (small objects)	Yes
Geo-Position	No	No	Yes	No	No
Cloud-solution for marker detection	No	No	Yes	Yes	No
SLAM	Yes	Yes	Yes	No	Yes

Which content can be used in AR and MR Apps?

AR and MR apps often use text, video (as clips or live-streams), 2D and 3D graphics (with and without animations), 360° panorama and Turn-Around-Recordings and audio data. Every data format related to smart devices, whether stored on the device or avaiblable via cloud services is generally possible. Dynamic contents out of data banks and other sources (Weather services, sport events, social networks, etc.) can also be added to the application.

Advantages of custom built AR Apps

• Development of individual and innovative solutions
• Optimized orientation of the features for the company’s own goals
• Unrestricted possibilities to make the own app sales efficient (branding of Content and App)
• Free choice of technology and configuration of technology
• Independent from AR Browsers (Availability of the server, etc.)