Developing AR Apps

Augmented Reality (AR) started with simple apps that offered either location-based information or visualization based on the scanning of 2D markers. As AR moves into the domain of sophisticated image processing, 3D rendering and interactions, there's a need for better AR authoring tools.

In general, 3D tools are complex and difficult to master. Creating high quality 3D models also require particular expertise. If AR has to take off, authoring tools must offer simple and intuitive workflows. This is already happening. The idea is to enable domain experts, even if they lack 3D modelling expertise, to easily create AR experiences. It's expected that the AR ecosystem will eventually offer tools and SDKs for developers at all levels.

Discussion

  • What sort of data can I use for AR content?
    Multiple data sources and multiple device targets in AR. Source: Curran 2016.
    Multiple data sources and multiple device targets in AR. Source: Curran 2016.

    AR content can include a variety of data input sources: CAD files and wireframes, 3D video, holographs, 360-degree images, 3D representations of real environments, and 3D sound. Increasingly, authoring tools will provide ways to integrate 3D design with AR. For example, 3D gaming engine Unity can import ARToolkit for tracking purpose. In 2015, CAD vendor PTC acquired AR authoring system Vuforia.

    Authoring systems are also capable of exporting AR content targeting a variety of devices: smartglasses, smartphones, tablets, head-mounted displays, etc.

  • What's a typical AR app development workflow?
    A typical AR interface development workflow. Source: Christ et al. 2018, fig. 3.
    A typical AR interface development workflow. Source: Christ et al. 2018, fig. 3.

    For any project, it's customary to identify the problem, propose a solution or approach, list the requirements, study technical feasibility, select suitable tools and technologies, etc. Specifically for AR app development, there are some unique things to consider:

    • Data acquisition: In medical field, for example, data can come from CT or MRI scans. In retail, data may come from printed catalogues. In engineering, data may come from sensors. To build accurate models, data may need to be cleaned and preprocessed.
    • Modelling: 3D models are generated based on the data. If models are already available, then they may be simply imported into AR authoring environments.
    • Interface development: Developer defines the scenes and their transitions. Models are imported into scenes. Within scenes, user interactions are enabled, such as clicking, zooming or rotating 3D models. The AR experience is then tested within the development environment and later on target devices.
  • As an AR app developer, what tools should I adopt?
    A comparison of some AR SDKs. Source: Bryksin 2017.
    A comparison of some AR SDKs. Source: Bryksin 2017.

    Because 3D content is an important aspect of AR, and many game engines already work with 3D content, game engines are commonly adopted by AR developers for authoring AR experiences. Top game engines include Unity and Unreal Engine. The development process can be simplified by using an AR SDK. Popular AR SDKs can be imported into game engines. For example, Vuforia can be used from with Unity.

    When selecting an SDK look for these capabilities: licensing, supported platforms (iOS, Android, Universal Windows Platform), target devices (phones, tablet, smartglasses, HoloLens, etc.), supported game engines, recognition (cloud or on-device), 3D tracking, geolocation, Simultaneous Localization and Mapping (SLAM)...

    AR SDKs are many: PTC Vuforia, Apple ARKit, Google ARCore, Wikitude, Kudan, MaxSt, EasyAR, and more. Some SDKs may be better at marker-based AR while others may be better at SLAM. Some SDKs such as Wikitude and Vuforia allow the use of other SDKs such as ARKit.

  • Do I need specific math skills to be an AR developer?

    When creating basic AR experiences, the use of AR authoring tools is more important. Developers can get by with basic math skills. For example, in Unity, you can scale or rotate 3D models by simply changing some settings. With Google's Sceneform SDK and its runtime API, developers can avoid learning the whole 3D development stack. With Blippar SDK, object recognition is made easy by calling computer vision APIs.

    For more complex AR experiences, knowledge of linear algebra, vectors and matrices will help. To build or manipulate 3D models, 3D math, 3D design, rendering and UX design become important. When handling sensor data, you need to know about transforms and filters. These are the same skills that game developers possess. Thus, game developers could move into AR development roles.

    Increasingly, Machine Learning is being used to process sensor data including live video feeds. This would mean that knowledge of statistics and ML models is relevant.

  • Could you share some tips for creating compelling AR apps?

    John Fan, CEO and Founder of Kopin, gives us five rules for building great AR:

    • Humans first: User comfort must come before technology. To wear something, users must be convinced of its real value.
    • Physical world first: Too much virtual content can overwhelm users. Use overlays only to complement real-world experiences.
    • Maintain situational awareness: All senses must continue to be in touch with the real world. Physical reality cannot be replaced.
    • Voice is the new touch: Keyboards and touch screens are compromises. Voice should be used effectively for command and control.
    • Balance design with benefits: Don't overdesign. Remove unnecessary features. Design for specific AR benefits.

Milestones

2014

Google announces Project Tango. It enables AR by its ability to map its surroundings using sensors, motion tracking camera, 3D depth sensing, depth perception and area learning. It's available in limited devices that meet its hardware requirements. With the coming of ARCore in 2017, it's discontinued in 2018.

Jun
2017

Apple announces its AR platform named ARKit. This uses Visual Inertial Odometry (VIO). Using camera and motion sensors, it marks the environment with a bunch of points and tracks them as the user moves around.

Aug
2017

Google releases ARCore, an AR SDK for the Android platform. Unlike Google's Tango, ARCore can work on many Android phones without any specialized hardware. ARCore works with Java/OpenGL, Unity and Unreal. It's capable of motion tracking, environmental understanding and light estimation. While not as advanced as Tango, ARCore has potential to reach mass market.

Oct
2017

Unity 2017.2 is released with support for Vuforia, ARCore, ARKit and Windows Mixed Reality immersive headsets.

May
2018

ARCore 1.2 enables multiplayer AR across both Android and iOS. It's capable of wall detection. Cloud Anchors allows virtual objects to be placed in 3D space and synchronized to other devices via the cloud. Sceneform SDK is also released.

References

  1. 24Seven Talent. 2018. "How to Land a Job in Augmented Reality/Virtual Reality (AR/VR)." 24Seven Talent Blog, February 27. Accessed 2018-05-24.
  2. Amadeo, Ron. 2018. "Google’s ARCore 1.2 enables multiplayer AR across Android and iOS." Ars Technica, May 9. Accessed 2018-05-24.
  3. Betters, Elyse and Cam Bunton. 2017. "Google Tango explained: What could it do and why was it shut down?" Pocket-lint, December 15. Accessed 2018-05-24.
  4. Blippar. 2017. "Blippar - Where AR and AI Meet." Blippar, YouTube, October 4. Accessed 2018-05-24.
  5. Bryksin, Gleb. 2017. "Best Tools for Building Augmented Reality Mobile Apps." Upwork, July 4. Updated 2018-05-21. Accessed 2018-05-24.
  6. Burke, Dave. 2017. "ARCore: Augmented reality at Android scale." Android Developers Blog, August 29. Accessed 2018-05-24.
  7. Christ, Roxie, Julien Guevar, Matthieu Poyade, and Paul M. Rea. 2018. "Proof of concept of a workflow methodology for the creation of basic canine head anatomy veterinary education tool using augmented reality." PLOS, April 26. Accessed 2018-05-24.
  8. Curran, Chris. 2016. "How will people create content for augmented reality?" PwC Blog, May 16. Accessed 2018-05-21.
  9. Fink, Charlie. 2018. "Five Rules For Doing AR Right." Virtual Reality Pop, March 13. Accessed 2018-05-23.
  10. Jasani, Tejas. 2014. "The top 10 engines that can help you make your game." VentureBeat, August 20. Accessed 2018-05-24.
  11. Lian, Alex. 2017. "Unity 2017.2 is now available." Unity Blog, October 12. Accessed 2018-05-24.
  12. Parikh, Karan. 2016. "What mathematics do I need to understand for AR and VR development?" Quora, December 13. Accessed 2018-05-24.
  13. Prabhu, Sanket. 2017. "Augmented Reality SDKs in 2018: Which are the best for Development." AR reverie, December 21. Accessed 2018-05-21.
  14. Robertson, Adi. 2017. "Breaking down Apple’s new augmented reality platform." The Verge, June 6. Accessed 2018-05-24.
  15. ThinkMobiles. 2017. "8 best Augmented Reality SDK for AR development in 2018." ThinkMobiles, January 20. Updated 2018-05-15. Accessed 2018-05-21.

Further Reading

  1. Taywade, Gaurav. 2017. "Best available SDK for developing AR applications." ARVR Journey, November 21. Accessed 2018-05-21.
  2. Curran, Chris. 2016. "How will people create content for augmented reality?" PwC Blog, May 16. Accessed 2018-05-21.
  3. Bilgili, Dogacan. 2017. "Introduction to Vuforia on Unity for Creating Augmented Reality Applications." Envato Tuts+, July 13. Accessed 2018-05-24.
  4. Herrera, Diego. 2018. "Introduction To Unity® For Absolute Beginners | 2018 ready." Udemy, May. Accessed 2018-05-24.

Article Stats

Author-wise Stats for Article Edits

Author
No. of Edits
No. of Chats
DevCoins
User avatar hemanthSK
2
0
986
Avatar of user arvindpdmn arvindpdmn
1
0
3
DevCoins due to articles, chats, their likes and article hits are included.
1038
Words
2
Authors
3
Edits
0
Chats
3
Likes
4814
Hits

Cite As

Devopedia. 2018. "Developing AR Apps." Version 3, May 24. Accessed 2023-11-14. https://devopedia.org/developing-ar-apps
Contributed by
2 authors

Last updated on
2018-05-24 11:45:12
mobile wearable sdk game engine

See Also