2017 looks set to be the year that augmented reality (AR) moves firmly into the mainstream. As the next big technology trend, it looks set to have a massive impact on user experience and, by extension, user experience design. The good news is that augmented reality has the potential to solve user problems that have challenged UX designers for years, like high interaction costs and low engagement. On the flipside, augmented reality also brings challenges for designers. This post will look at how augmented reality is affecting UX, and how UX designers can rise to the challenge of designing engaging, conversion-friendly, augmented user interfaces.

What is Augmented Reality?

Although it’s been around in some form since the early 90s, the term ‘augmented reality’ can still cause a little confusion. In a nutshell, augmented reality is technology that combines inputs from the real world with programmed components. These programmed components have to interact with real world data in some way, changing as the real inputs change. Whereas virtual reality (VR) shuts users off entirely from the real world and places them in an entirely fictional alternative, augmented reality adds a programmed layer over actual reality to create a third, dynamic level of augmented experience. So that hologram video of Michael Jackson dancing? Plenty of people claimed it was augmented reality, but they were mistaken. The hologram Jackson did not respond to real world inputs and was incapable of dynamic change based on those inputs. If anything, as Angie Li and Therese Fessenden write for NN Group, the real world dancers augmented the fiction, rather than vice versa. Examples of AR that the majority of users have at least heard of, if not used, are things like Snapchat, Pokémon Go and Microsoft’s HoloLens.

How Does Augmented Reality Work?

Designing delightful augmented user experiences requires an understanding of the technology’s functionality. A device’s numerous sensors—for example the GPS, camera and compass on a smartphone—feed inputs into a pre-installed software application on the device. The device interface then independently responds by adding sensorial enhancements to the scene. Therefore, an augmented reality interface is an example of what’s known as a ‘non-command user interface’; the interface acts and reacts without specific user commands. It’s important to point out that even if you have the latest AR-friendly smartphone, that doesn’t mean you’ll be seeing augmented reality elements everywhere you go. The technology also requires the AR software to be fired up, and a real world ‘trigger’. These three elements combine to make the augmented user experience possible.

What Does Augmented Reality Mean for UX Designers?

Obviously, the implications of augmented reality for user experience are enormous. The technology has the potential to completely revolutionize the way users interact with devices. From designing user flows for reactive interfaces to imagining users existing in a screenless world where everything has the potential to become an interface on command, the changes to UX design will be fundamental. Exciting, right?! But also terrifying. What with its unique blend of psychology, research, interaction design and software development, user experience is complex enough already. UX Designers need to futureproof themselves and ensure they’re ready for the opportunities and challenges that are already knocking on their doors.

UX Opportunities of Augmented Reality

Augmented reality is already changing users’ experiences of brands and digital platforms. At first glance the two most well-known examples, Pokémon GO and Snapchat, use AR in a pretty light way. After all, catching pokémons and jazzing up selfies with a flower crown hardly looks like ground-breaking UX. But let’s look at the numbers: Within a week of its release in 2016, 65 million people worldwide were playing Pokémon GO, and Snapchat valued at $24 billion when it went public in March (although share value later dropped and leveled out). Snapchat and Pokémon GO’s success is due to the fact they are using AR to provide users with a unique, customizable experience that keeps them hooked. The reasons for the stickiness of AR experiences aren’t hard to define. AR decreases interaction costs, reduces a user’s cognitive load, combines multiple sources of information, and minimizes attention switches. Pretty much the Holy Grail of UX. [pullquote]AR decreases interaction costs, reduces a user’s cognitive load, combines multiple sources of information, and minimizes attention switches. Pretty much the Holy Grail of UX.[/pullquote] It’s therefore no surprise that we’re seeing AR taking hold beyond social media and gaming. The technology is already being exploited by forward-thinking marketing departments as a great branding opportunity. Take Pepsi as an early-adopter: In a 2014 the soda company ran a campaign on London bus stops which saw AR technology transform a normal, boring bus shelter wall into a screen full of asteroids or tigers on the loose. Maybe the campaign didn’t sell Pepsi right there and then, but in terms of branding it was a powerful move. UX Designers will work closely with marketing teams to test and refine augmented experiences that reinforce brand image and engagement. Conversions are another area where the potential of augmented reality is already being put to the test. Let’s take the example of a consumer shopping for a table. The simplified user journey might be: Research tables online, go to stores, take photos on mobile device, look at the photos in their home…And then uh-oh, it’s impossible to know which table is right just from some crummy photos taken in store. Would-be consumer is frustrated, fails to convert. Stores such as Ikea are using augmented reality to solve this. Using an Ikea app, users can point their mobile at an area in their home, and generate an augmented view of the same scene complete with a device-generated piece of furniture. The same works for buying pretty much anything online, from cosmetics to clothing. Consumers have already demonstrated their desire for this kind of experience: according to stats from MavenEcommerce, the type of AR app that people are most interested in for online shopping are:

• Virtual dressing room - 88%
• Shoe sampling - 87%
• Virtual furniture sampling - 86%
• Interactive vehicle manual - 75%

Allowing the user to see the object in context solves the obstacle to conversion and everyone is happy. In the not-too-distant future augmented reality will probably provide even more out-there opportunities to improve user experiences. Think about augmented presence meetings and teleconferences, where a colleague from some place else on the planet, just like AR entrepreneur Meron Gribetz does in this TED talk. Devising the UX flows, interactions, gestures and UI animations for this kind of technology will fall to the augmented UX designer. Whereas now UX designers work with interactive prototyping tools to model and test realistic user interfaces, in the future they’ll combine UI prototypes with 3D prototypes from tools like Microsoft’s HoloLens. It could be that existing prototyping tools will introduce AR capabilities to keep up with the trend.

UX Challenges of Augmented Reality

But, as with any seismic tech shift, there will be challenges for UX designers designing for augmented reality. Let’s take a look at those challenges.

Terminology

Even before UX designers get started in AR, they’ll need to get to grips with a whole lot of new terminology. Familiarising yourself with terms such as modulated reality, HMD and HUD is a good starting place for meeting AR challenges head on, and starting to think realistically about augmented user experiences.

Overkill

This is going to be crucial to AR’s long-term success. To see what we mean by AR overkill just check out this video by Keiichi Matsuda, which presents a vision of an over-augmented user experience. The video is great, but would you really want to live with all that visual noise in your head all day? UX designers must avoid overwhelming users with useless or decontextualised information, instead using AR to add value in certain contexts.

Hardware and Physical Comfort

UX designers are used to designing experiences to reduce eye strain or thumb strain. But augmented reality will require consideration of other physical factors, such as arm strain: users will not walk around all day holding their mobiles at eye level. The comfort of wearables will become more and more important to AR adoption.

Conditions and Context

The environmental challenges thrown up by augmented reality are going to be huge. When designing for desktop experiences, UX designers are working with a relatively limited set of variables regarding the conditions a user might be in. With AR, a user could be absolutely anywhere, looking at anything, under any conditions, and still expect a stellar augmented experience. UX designers will have to run comprehensive user tests on software use in light and shady conditions, different weathers, interiors and exteriors, in motion or still. User testing is going to have to adapt substantially.

Safety

Users' physical safety is also an issue. Pokémon GO designers ran across this when there emerged reports of players being so absorbed in the game that they were hit by cars or walked off cliffs. That’s probably the acme of bad user experiences, so augmented reality experiences will have to be designed that minimize dangerous outcomes. These challenges mean that UX designers need to have a deep understanding of users’ expectations around the new technology before they start to design experiences to match.

Best Practices for AR UX Design

The field of designing augmented reality user experiences in still in its infancy, but there are many best practices that UX designers can start thinking about now.

Always Think About the Environment

Whether the AR campaign will take place on the side of a bus shelter like Pepsi’s, in user’s homes or on the streets, UX designers have to design interactions for those conditions, and test them fully in those conditions. Rob Manson from AR UX has helpfully broken down physical user scenarios into 4 main categories:

• Public, in which the user uses their whole body to interact with the software
• Personal, in which the user uses a smartphone in a public space
• Intimate, in which the user is sitting with a desktop and is not really in movement
• Private, in which the user has on a wearable

UX designers will have to define user journeys for the relevant physical scenarios and define how the interface will react to each.

Make it Comfortable for Users

Users hold their devices in the most comfortable, least effortful way they can. If you’re designing an AR experience for a handheld device rather than a wearable, you have to take that physical comfort factor into account. Make sure you’re up to speed with device use archetypes: for example, users will hold their device for longer when seated, or if they’ve got the device at chest level.

The Interface Should be as Automatic as Possible

The whole idea of a non-command interface is that it operates alone. Augmented Reality experiences should be designed to need as little physical input from users as possible. This makes sense, because if users are looking through the device screen at an augmented picture, it’s going to be hard for them to use gestures at the same time. Voice control is the obvious answer to this, and UX designers will have to start designing more voice interaction flows, as with Siri or Alexa.

Start Designing AR Experiences Now

As in right now. UX designers need to start understanding and trying out AR principles as soon as possible if they’re not to be left behind by the AR wave. Luckily, platforms like Coursera have some good introductory online courses, and some AR tools themselves also offer toolkits, such as Microsoft’s HoloLens.

How AR is Changing User Experience Design, Final Word

Augmented reality has the potential to give users the kind of experiences they crave: exciting, useful, usable and meaningful. It has the potential to marry user needs with business goals and create something digitally delightful in the process. But that kind of success story depends on UX Designers meeting the inevitable challenges thrown up by a new, exciting technology. UXers can do that by getting up to speed with current AR tech and, as always, empathising with the user to give them not only what they need, but also what they want.