Exploring Augmented Reality and Experiential Design
Summary
Research and exploration of scenarios in which augmented reality (AR) could prove an effective component of experiential design. Creation and deployment of photo.ARt, an immersive augmented reality app.
Problem Statement
With games like Pokemon Go helping to make AR more mainstream, the realm of possibilities for augmented reality technology continues to expand. For the purposes of this project, I wanted to research the impact of AR in the context of acting as a component of an experiential design strategy, then explore scenarios in which AR can be implemented in settings that could lead to positive enrichment experiences, such as improving user engagement, affecting mood, providing entertainment, and impacting task efficiency.
Project Goals
- Research AR technology to examine the potential benefits as they relate to gratification, gamification, and task efficiency
- Explore several experiential design scenarios in which AR could provide positive enrichment experiences
- Build and deploy an AR application using image targeting, audio, video, and immersive 3D environments to serve as a proof of concept
Project Duration
My Role
My Contributions
Software Used
Exploring your neighborhood while chasing Pokemon, checking out what that new sofa would look like in your living room—there's no denying that augmented reality is cool! But outside of the sheer novelty of AR, what are some practical, real-world applications of the technology that could have a positive impact on individuals? Let's look a variety of circumstances in which AR technology can be utilized, and examine the effect they have on the human experience.
Gratification
One study looked at the motivations for wearing augmented reality smart glasses (ARSGs), and their correlation to user gratification. Researchers found that a user’s life efficiency (i.e., the belief that ARSGs can help them do certain tasks more efficiently), enjoyment, and desired enhancement of reality influenced motivation for wearing ARSGs. 1
Gamification
Researchers in Japan uncovered the positive impact of AR on patients suffering from severe social withdrawl (hikikomori). When the game Pokemon Go was introduced to patients, researchers noted improvements in social partcipation and receptiveness to family support, suggesting future consideration of AR games for therapeutic treatments. 2
Task Efficiency
Another study compared urban navigation using a system consisting of assisted GPS (AGPS), location-aware range queries, and AR, with standard plane map navigation systems and uncovered the benefits of AR-enhanced navigation, which included improvements in time savings, usability, enjoyment, accuracy, and ease of determining direction. 3
Experiential Design
When looking at experiential design and its effect on participants, researchers in Germany examined experiential design as part of a museums' branding strategy and, through simulations, uncovered a positive impact associated with with experiential design, especially among "low-involved" participants as it relates to "branding relevant behavior." What does this imply? Experiential design can lead to positive impressions of the experience, alter perceptions of museums, even lead to repeat visits and positive word-of-mouth advertising. 4
Let's focus on opportunities for the photo.ARt app concept as it pertains to the research findings noted above. When used as part of an experiential design strategy, the app would be able to surface multimedia content, such as video, audio, 3D objects and environments, and characters that can interact with the user simply from targeting everyday items such as artwork, photos, posters, business cards, QR Codes — even 3D objects.
Outside of the mainstream implementations of image-targeted or geolocation-based AR, such as mobile gaming, fashion, and home decor, there are several other industries where this technology could provide a positive impact. For anyone who has been to a museum and rented an audio guide, the benefits of AR should be obvious. Resorts and theme parks could add AR apps to their offerings, encouraging exploration of the property and providing entertainment for youth programs.
Exhibit Design
Museums & Galleries
Entertainment
Resorts & Theme Parks
Marketing
Conferences & Events
With a deeper understanding of the manners in which AR technology can impact humans and their everyday, ordinary experiences, I set out to develop an app to explore how that could look. I drew inspiration from scenerios uncovered in the "Exploring Opportunities" section. I researched what was possible in my limited time, and decided to produce an app that could incorporate not only my love of design, photography, and filmmaking, but my desire to wow and inspire my audience. App development was fairly straight forward, but it took a few weeks. It was broken up into a bit of preliminary creative prep work in Photoshop, Premiere, and After effects, followed by image targeting in Vuforia, modeling the environments in Unity 3D, and deployment to iPhones.
Photo editing and post production
- Processed the photographs that were used as image targets.
- Created graphics and textures in Photoshop.
- Edited video footage, and added music and sound effects, to generate four short videos that were used in the app.
- Added transitions and visual effects to the videos in After Effects.
Vuforia Image Targets
- Created a Vuforia developer account.
- Added a variety of images to the database to determine which would best serve as image targets.
- Set up image targets in Vuforia that trigger the augmented reality environment.
Unity 3D app development
- Imported image targets from Vuforia.
- Imported videos, graphics, and textures.
- Modeled the 3D environments using a combination of created assets, found assets, graphics, and textures.
- Wrote C# code to control video playback when image targets are active or inactive.
Build in Xcode and deploy to iPhones
- Built the project in Unity 3D for iOS devices.
- Built the project to an app in Xcode.
- Performed a bit of troubleshooting to actually get it to work.
- Deployed the app to both an iPhone 7 and iPhone XR.
When the the app was complete, I enlisted our daughter to perform a demo in front of family and friends, most of whom hadn't yet seen it. I filmed the demo live with a Lumix G85 and a Rode video mic to get a sense of the experience, then grabbed a separate screen recording of the experience on my iPhone. I combined the two videos below for a multii-view take on the AR experience. Enjoy!
This demo served as a great proof of concept for what the app was capable of. There are couple features, however, that I would have liked to have added had time not been a factor. The ability to interact with the environment would be the next logical step in the progression of this project, which is what would really generate interest in the commercial space.