Augmented Reality App and AR Browser
AR Technologies can be used in various industries and business sectors. Due to the variety of different types of application and external conditions, two typical approaches for the development of AR-Apps have been established. For one thing, custom made AR Apps can be produced with complete configurability of every feature in order to achieve ideal solutions for every challenge. For another, already existing AR Browser Apps, that are seen as some sort of publishing platform and offer standardized features (sometimes even as an Editing-Tool). Even though AR-Browser Apps permit their producers a comparatively easy access into the AR-Content-Progressing and Publishing, due to extensive automation and strictly defined templates of features, the possibilities of using more complex AR-Functions however is mostly impossible and own contents can only be used restrictively to be sales efficient and advertised (less branding possibilities). Custom made AR-Apps however can be realized in a very complex manner, accomplish even difficult technological challenges and can be advertised and published as an independent application. In the following, both approaches will be explained.
Augmented Reality Apps
What is an Augmented Reality App?
AR Applications offer their users additional information on their location, objects (e.g. products) in their immediate environment or enable the insertion of virtual objects on the display respresenting their real environment. In doing so, AR Apps can be used in almost every industry and can also be beneficial for entertainment, training, improvement of productivity, communication, sales increase. Augmented Reality can appear as various display formats, such as simple insertion of texts, videos and audio data, 360° Panorama images, 360° turn around images, animated 2D and 3D contents and also linear and interactive mechanisms for user instructions and interactions. The AR App can be used as simple client applications on the terminal device or more complex client-server applications on linked systems. As some properties of AR solutions are highly dependent on the purposse of use, AR Apps are usually made-to-measure, individual products based on Augmented Reality SDKs (Software-Development-Kits, etc.) and other basic technologies (e.g. Apple ARKit, Google ARCore, vuforia, etc.).
Which AR Apps exist?
Augmented Reality Apps exist in various industries and divisions. Due to the variety of the existing apps and the fact that they are published on various platform (Smartphone, Tablet, Computer, Headset, etc.) and operation systems, it is hardly possible to show a complete list of AR-Apps. In the following we have listed some of the best-known examples. Furthermore, more detail and information in the category “What is Augmented Reality?” is available (the difference between AR, MR and VR will also be explained).
- LEGO Digital Box
Running in a Kiosk-System at the Point of Sale (LEGO-Stores) to communicate the product and support sales - IKEA App
Running as a smartphone App: addition to the IKEA catalogue (AR-Print/Interactive-Print) to preview the products - Pokémon GO
Running a s a smartphone App: most famous representative for AR-Games
Hint: The mentioned examples are AR-Apps developed by other companies, not ours.
How do Augmented Reality Apps work?
In order to realize reality visually and expand it due to its context, the app needs to “understand” the situation or environment in which it is being used. Various technologies (accessable as AR SDKs) are used for this – most of them are basic technologies using 2D Image marker recognition. More complex SDKs even offer 3D object recognition, SLAM or Visual Inertial Odometry (VIO) processes and might even allow GEO-Positioning via GPS. Most of these approaches analyse the picture of the final device and add audio-visual contents (e.g. Texts, Pictures, Videos, even 2D or 3D animations, etc.). The expanded reality is then showed on the display. As the complete process is happening simultaneously, the user thinks that the virtual additions are right on their screen and therefore a part (if even only on screen) of their reality.
Overview of the details of different Tracking-Approaches:
2D Image marker / Tracking marker
The term Image marker or also tracking marker usually describes an Image, which is usually stored in an AR-App or the related Cloud Service for comparison purposes, and therefore already known. The comparison is usually made via the analysis of light and dark in the picture and provides so called “Image-Features” in the form of contrast points. During the process of an application, the active camera picture is analysed and everything visible on the video picture is checked and compared on the Image markers (and their contrast point pattern). If this analysis matches, additional information can be assigned to the identified image marker and in addition it is possible match the position and rotation of virtual contents with the image marker based on the camera’s perspective. Therefore augmentations/holograms can be shown accordingly. Image markers are so to say a little bit more than QR-Codes. They do not only refer to additional data but also help to initialize a 3D coordination system in order to super impose the camera’s view.
3D Objekt recognition
Not only can 2D markers be replaced with the 3D object recognition, but also more complex applications can be realised. Instead of storing a 2D image reference, a 3D volume reference in the form of a Point Cloud is stored in the system. A camera repeatedly films the real object, analyses the video image and compares it to the 3D reference. To simplify this: the image analysis tries to copy the contrast points and edges of the video image and use them as a comparison to the stored 3D data basis. If this complex analysis identifies a known object in the camera picture, additional and linked information (text, video, 2D and 3D graphics, etc.) can be augmented on the display correctly in the perspective of the camera’s view. A huge advantage compared to the 2D image marker variation is, that the real object can be seen from all sides and steadily augmented.
SLAM (Simultaneous Localization And Mapping)
Solutions for SLAM (simultaneous location and mapping) features usually involve methods for constructing or updating a map of an unknown environment while simultaneously keeping track of the smartphone’s location within it. Initially, SLAM algorithms have been used in the context of autonomous robots. However they are also of high importance for AR and MR scenarios. AR and MR applications usually develop their full potential when the user’s device and app know as much as possible about the real environment and therefore the context. With the help of SLAM AR and MR solutions have the possibility to find their ways around in unknown territory and therefore collect information on environment and context of the scenery – an advantage that makes new ways of application possible, even without the use of reference markers.
Visual Inertial Odometry (VIO)
Processes that use the data of cameras (pictures), position and acceleration (Gyroscope and acceleration sensors) to identify the position, orientation and movement of a device in space are called “Visual Inertial Odometry”. Further usage of data leads to similar results as in the SLAM process and in addition offers a very precise comparison of lengths, sizes and distances. Therefore, virtual objects can be shown in realistic size dimensions without standardized tracking markers.
GEO-Positioning / Location Based Service (LBS)
As a “Location Based Service”, AR Apps show relevant information on the current location of its user. In order to do this, the users position is analysed via GPS and stored content will be loaded in its defined environment. Next to details on close buildings, shops and sights, virtual objects can be shown as virtual objects as 3D models or interactive elements can be implemented. A well-known example for an intensive use would be the App Pokémon GO, where the device/app tracks the users location, shows it as part of a map and adds virtual elements (Pokémon, Arenas, etc.). The users position and the position of a virtual addition is analysed and transferred. This creates an illusion of Pokémon around everyone’s neighbourhood.
What is an Augmented Reality SDK?
AR SDKs offer advanced technologies and tools to create AR solutions for the AR App developers. They contain important fundamental functions, such as image or object recognition, SLAM and different other software algorithms and partly render engines to show the content. Best-known SDKs would be the ARKit (Apple – after metaio’s Akquise), ARCore (Google), vuforia (PTC – used to be Qualcomm’s QCAR), Wikitude, MaxST, Kudan and ARToolKit, whereas the quality and types of software packages differs in their AR Technology elements.
ARKit | ARCore | Wikitude | vuforia | Kudan | |
---|---|---|---|---|---|
Operating system | iOS | Android | iOS / Android | iOS / Android / Windows UWP | iOS / Android |
Maximum distance to marker (detection / tracking) | - | - | 0.8m / 3m | 1.2m / 3.7m | 0.8m / 3m |
Tracking quality for static marker | 8/10 | 2/10 | 10/10 | 10/10 | 9/10 |
Tracking quality for moving marker | 6/10 | 2/10 | 8/10 | 6/10 | 6/10 |
2D Tracking | Yes | Yes | Yes | Yes | Yes |
3D Tracking | No | No | Yes | Yes (small objects) | Yes |
Geo-Position | No | No | Yes | No | No |
Cloud-solution for marker detection | No | No | Yes | Yes | No |
SLAM | Yes | Yes | Yes | No | Yes |
Which content can be used in AR and MR Apps?
AR and MR apps often use text, video (as clips or live-streams), 2D and 3D graphics (with and without animations), 360° panorama and Turn-Around-Recordings and audio data. Every data format related to smart devices, whether stored on the device or avaiblable via cloud services is generally possible. Dynamic contents out of data banks and other sources (Weather services, sport events, social networks, etc.) can also be added to the application.
Advantages of custom built AR Apps
- Development of individual and innovative solutions
- Optimized orientation of the features for the company’s own goals
- Unrestricted possibilities to make the own app sales efficient (branding of Content and App)
- Free choice of technology and configuration of technology
- Independent from AR Browsers (Availability of the server, etc.)
Augmented Reality Browser and CMS
What is an AR-Browser?
Beside the possibility of developing individual AR-Apps, AR contents can also be published on existing platforms of third parties and partly revised editorially- Existing Applications (e.g. BlippAR, Wikitude or Aurasma) are defined as AR plattforms and are called AR-Browsers due to their functionality. They not only contain standardized AR features but also developer tools to easily create and manage contents (Content Management Systems – CMS). AR Browsers are applications for Smartphones (e.g. iPhone) and Tablets (e.g. iPad) or other AR/MR terminal devices (compare Augmented Reality Hardware) that show additional information on the display of its user, based on 2D image markers, 3D object markers, SLAM methods or GPS tracking. iOS (Apple iPhone/iPad) and Android (Samsung, HTC, LG, Motorola, etc.) – the two famous mobile operating systems – are mostly supported by existing AR Browsers.
Which AR Browsers exist?
Since 2008, several AR Browsers have developed on the market and are mostly free to use. Some well-known representatives are listed:
- Wikitude (10/2008)
- Layar (06/2009) and BlippAR (10/2011) – both are part of the BlippAR Group
- junaio (09/2009) – was stopped after Apple took over (former metaio)
- Aurasma (05/2011)
How do AR Browsers work?
AR Browsers can be seen as a platform for an easy and comparably reasonable publishing of AR contents. While doing so, simple AR functions, such as the overlapping of texts, 2D images, videos and simple 3D pieces, are being offered as modular features for the client. Producers but also amateurs can create contents with the available tools and connect coordinates via 2D image markers or GPS. Both contents and image markers have to be available online. That is how one can create and publish “worlds” at Wikitude, “layers” at layar and “auras” at aurasma. Every client receives his own “channel” (similar to the URL in web browsers) and can/has to use this channel as an entrance to his AR-Experience and publishing.
Among the to be installed AR-Browser App, end users have to find access to the channel of the AR-Content-Provider. To make it as simple as possible to find access to a channel, “Search”, “Favourites”, and partly Visual Search Functions (Image photographs, result selection) have been installed. If a world/layer/aura is being started, the AR-Contents of each Channel-Provider load and can be explored.
Properties of AR Browsers
- Cost efficient development, publication and maintenance of AR Contents
- Spreading of established AR Browsers and their reputation among end users
- Promotion via established channels
- Creation and maintenance of AR Contents by “Amateurs” is also able after training