Designing Interactive Web Based
AR Experiences
Reza Ali | @rezaali | WebXR | Daydream | Google | FITC Toronto 2018
Why smartphone AR?
Accurate 6-DOF motion tracking
Compositing the virtual and real
Basic world understanding:
• Feature points & marker detection*
• Horizontal & vertical plane detection*
• Basic illumination determination
The future of augmented reality
sounds a lot like the web
WebARonARCore WebARonARKit
WebARonARCore
github.com/google-ar/WebARonARCore
WebARonARKit
github.com/google-ar/WebARonARKit
three.ar.js
github.com/google-ar/
three.ar.js
We’ve been building prototypes to help explore:
• Impactful use cases (understanding, vibes)
• design principles (how, why, what, when, where)
• diegetic interfaces (XR UX)
• interaction design (raycast everything)
• rendering techniques (sdf font rendering)
• developer workflows (desktop prototyping)
WebAR-Article
🤔
How long did it take?
What went into it?
How was this crafted?
How was this design?
Where is the code?
ES6
Minimizing the use of new
Scratch matrices, vectors, quaternions
ES6 imports / exports to avoid pollution
Used [raw, image, glslify, json] loaders
Serving & reload via webpack-dev-server
eslint to catch mistakes and garbage
Low Poly [Obj & Mtl]
Performant ⚡
Public & Remixable
CC-BY License 🙂
Beautiful & Playful
Baked AO 😋
poly.google.com poly.google.com
Converted .obj to .drc
Astronaut From Poly
185k > 9.5k ~20x 🤓
Astronaut From NASA
3.1M > 121k ~26x 😎
https://github.com/
google/draco github.com/google/draco
Responsive Layout via Bootstrap
CSS Styles via Bootswatch (Paper Theme)
three.js (shadows, lights, shaders, math)
Model sourced from poly.google.com
Model compressed ~20x by Draco
Desktop AR Prototyping Environment 💡
Polyfills AR api
🔥 RAY MARCHES 🔥
virtual scene (SDFs)
GPU Hit Computation
GPU > JS Encode float
Simple 1 line include
Rapidly iterate UX & UI
Desktop AR Prototyping Environment
Show off
Searching Reticle
State
Show off
Tracked Reticle
State
Reticle Searching State
Reticle Tracking State
Reticle Fade Out
AR Flow
Show off AR
Controls
Production
AR Controls (Debug View) AR Controls
AR Transition (0.25x Speed)
Entering & Exiting AR
1. Remove model
2. Remove SDF grid
3. Remove HUD
4. Fade out sphere
5. Fade in HUD
6. Enter AR flow
Entering & Exiting AR
1. Zero window scroll
2. Pop off canvas
3. Remove all content
4. Add canvas back
5. Add fullscreen CSS
6. Reverse for exiting
WebAR-Article
github.com/google-ar/
webar-article
Annotations
Animations
Interactions
ARControls
Poly API
Animation Test
Performance Test
SDF Font Rendering
Proximity Based Annotations
WebAR-Place
Microsite Design
Conclusions
Design Guidelines
designguidelines.withgo
ogle.com/ar-design
WebXR Spec
https://github.com/
immersive-web/webxr
Getting Started
developers.google.com/
ar/develop/web/getting-
started
Thanks!
@rezaali
Off the record
Exponential growth
Are we there yet?
Novel hardware & software
Connecting the dots…

Designing Interactive Web Based AR Experiences