Augmenting Reality with FORMATION

Augmented Reality (AR) is getting a lot of attention with the recent launch of Apple’s Vision Pro and Meta’s affordable Quest 3 product. We have been keeping an eye on this space because of the potential in the context of our product.

The FORMATION app as it exists today is a map-based product that helps workers find their way in their workplace. It shows a lot of information on a map that is precisely geo-positioned on a 2D map. We also have support for buildings and floors, which enables using FORMATION indoors and outdoors. Of course the information we put on a map could also be super interesting in an augmented reality setting. We often talk about using the map as a lens to information. You could think of this as a simplistic form of AR. But imagine replacing the map with augmented reality—what possibilities would that unlock?

This was the question that we were looking to answer last year via two research projects.

HoloLens AR demo with Th Wildau

The first project we did with our friends at TH Wildau. We’ve been working with them in a few projects for several years. Last year, they built a demo application for Microsoft’s Hololens platform that allows users to complete tasks in FORMATION while wearing AR goggles.

Lars Schymik from TH Wildau created a short video that shows off the capabilities that AR offers. Particularly exciting for us is the concept of interacting with a QR Code tag simply by touching them with your finger. The AR goggles have a camera and when you wear them, QR Code tags become buttons that you can walk up to and press. And of course the general situational awareness is very useful too.

Also very interesting in his prototype is how the system keeps track of where you are and shares your position to FORMATION: we can use hololens to track the position of the user in real time. The waypoints that Lars is following, are created in FORMATION as regular points. You can see Lars walking through our office with the FORMATION app open in the AR view as well as a screen capture of FORMATION running on a laptop. As Lars walks around, the system tracks him and he checks off the waypoints until he reaches his destination. At the destination, he then interacts with the coffee machine to open a FORMATION task and completes it.

Visit the TH Wildau project page here .

Web AR demo

A second project we kicked off last summer was about coming up with an AR experience that doesn’t require special hardware. A few years ago, Pokémon-Go captured the imagination of millions of kids (and some adults) that were hunting Pokémon monsters with their smart phones. Simply by looking through the camera view of their phones, they can see tha augmented world with monsters and catch them.

This inspired us to see if we could replicate that experience with FORMATION. For this, we collaborated with two interns from CIEE . We gave them the task to create a prototype application using Web AR. Web AR is a standard that allows you to build Pokemon Go style applications where you can project simple objects in a 3D in the camera view.

Our two summer interns, Athena Rapinan and Donovan Rimer , worked on this for several weeks to overcome some challenges and managed to pull off a demo that’s very similar to what we did with HoloLens. Except it runs on a regular smart phone and does not require any special hardware.

The future of work

The future of work is here. Of course, the current generation of AR goggles is still a combination of too clumsy and awkward for daily use. They are big and clunky and the battery usage limits how long you can use them. However, as you can see from these two demos, the potential is there. And with Web AR, we might not even need any goggles. Access information and tasks about everything around you simply by walking around and interacting with things. We are very excited about the future. We’re looking forward to doing more things with AR in the future.