How I learned to stop worrying and love UX metrics

@tameverts
#PerfNow
How I learned
to stop worrying
and love UX metrics

What is the best UX metric?
How fast should it be?
How can we stay on track?

https://www.cnet.com/news/appliance-science-the-well-done-physics-chemistry-of-the-toaster/

TTFB DNS TCP Start render
DOM loading DOM ready Page load Fully loaded
TTI Resource timing Number of requests Bytes in
Speed Index PageSpeed Navigation timing DOM elements
DOM size Visually complete TTFP TTFMP

“The real thing we are after is
to create a user experience that
people love and they feel is fast…
and so we might be front-end
engineers, we might be dev,
we might be ops,
but what we really are is perception
brokers.”
Steve Souders

How do we measure
perception… at scale?

Is it happening?
Is it useful?
Is it usable?
Is it delightful?
https://developers.google.com/web/fundamentals/
performance/user-centric-performance-metrics

TTFB DNS TCP
TTI FCP FMP
FID OMG WTF

❑ Correlates to what users actually see in the browser
❑ Is easy to use and accessible right out of the box
❑ Recognizes that not all pixels and page elements are
equal
❑ Allows us to customize what we measure on specific
pages
The best UX metric…
#PerfNow@tameverts

A brief history
of performance metrics

Load Time
the time from the start of the initial
navigation until the beginning of the
window load event

BBC loses 10% of users for every additional
second it takes their site to load
Ancestry.com saw a 7% conversion rate
increase after improving load time by 64%
FT.com reduced desktop load times to
1.5s and mobile to 2.1s and increased
user engagement by 30%
@tameverts #PerfNow

Start Render
the time from the start of the initial
navigation until the first non-white
content is painted

Speed Index
average time at which visible parts of
the page are in the viewport

First Paint
First Contentful Paint
First Meaningful Paint
First CPU Idle
First Interactive

First Paint (FP)
Pixels first start to render

First Contentful Paint (FCP)
Text and graphics start to render…
BUT often catches non-meaningful paints
(e.g. headers, nav bars)
https://tinyurl.com/paint-metrics

First Meaningful Paint (FMP)
The paint after which the biggest
ATF layout change has happened
and web fonts have loaded
https://tinyurl.com/paint-metrics

https://speedcurve.com/blog/an-analysis-of-chromiums-paint-timing-metrics/

“The rendering pipeline is very complex, and
the timestamp should be the latest timestamp
the browser is able to note in the pipeline (best
effort).
Typically the time at which the frame is
submitted to the OS for display is
recommended for this API.”
https://w3c.github.io/paint-timing/#sec-terminology

Analysis of 40 top Alexa-ranked sites
95% of FP events occur before Start Render
85% of FCP events occur before Start Render
50% of FMP events occur before Start Render
https://speedcurve.com/blog/
an-analysis-of-chromiums-paint-timing-metrics/

❑ Correlates to what users actually see in the browser
❑ Is easy to use and accessible right out of the box
❑ Recognizes that not all pixels and page elements are
equal
❑ Allows us to customize what we measure on specific
pages
The best UX metric…

First CPU Idle
(formerly known as First Interactive)
Page is minimally interactive
Most visible UI elements are interactive
Responds to user input reasonably quickly

Time to Interactive (TTI)
(formerly known as Time to Consistently Interactive)
Displays useful content
Event handlers are registered for most visible elements
Page responds to user interaction within 50ms

Hero Rendering Times
H1
Largest Image
Largest Background Image
First Painted Hero
Last Painted Hero
https://speedcurve.com/blog/web-performance-monitoring-hero-times/

One-size-fits-most (not all)
Dependent on how page is built
Don’t work for image carousels and
popups
Hero times gotchas

http://lab.speedcurve.com/rendering/picker.php

It depends!
(You probably saw that coming.)

Deliver any content?
Start Render
Deliver significant
amount of content?
Speed Index, FMP
Deliver critical content?
Hero Rendering Timeshttps://speedcurve.com/blog/rendering-metrics/

Custom metrics
Measure performance with high-precision timestamps
Available in both synthetic and RUM (yay!)
https://www.w3.org/TR/user-timing/
https://speedcurve.com/blog/user-timing-and-custom-metrics/

how long does it take
to display the main
product image on my
site?

Time to First Tweet
The time from clicking the link to viewing
the first tweet on each page’s timeline
Pinner Wait Time (PWT)
The time from initiating an action (e.g.,
tapping a pin) until the action is complete
(pin close-up view is loaded)
Time to Interact (TTI)
@tameverts #PerfNow

First Click
First Scroll
First Key
First Input Delay
(only available with real user monitoring)

Great!
We have metrics.
Now… what to do with them?

Thresholds YOU create for metrics
that are meaningful for YOUR site
https://addyosmani.com/blog/performance-budgets/
Milestone timings (e.g. start render)
Quantity-based (e.g. image weight)
Rules-based (e.g. Lighthouse scores)

A good performance budget
should show you…
What your budget is
When you go out of bounds
How long you’re out of bounds
When you’re back within budget

Recommended UX perf budgets*
Metric How to track it Budget
Start render Synthetic & RUM 2s
First Painted Hero Synthetic 2s
Speed Index Synthetic 4s
First Interactive Synthetic 4s
First Input Delay RUM 10ms
@tameverts *See super important stuff on the next slide

Super important!
Look at your own data
Monitor your competitors
No sandbagging allowed
Take a step-by-step approach if necessary
Use synthetic and RUM (numbers may will vary)

Pro tips
Create budgets for your popular and regularly
changing pages
Review violations early and always
Compare before and after releases
Update budgets accordingly
https://www.zillow.com/engineering/bigger-faster-more-engaging-budget/

1. There is no unicorn metric
2. Use your eyes
3. Don’t just look at averages and medians
4. Validate your metrics by correlating them with business
and engagement KPIs
5. Set targets (e.g., performance budgets)
6. Make yourself accountable (e.g., alerts)
7. Don’t be afraid to investigate new metrics!
@tameverts #PerfNow

thanks!
@tameverts
speedcurve.com/blog

How I learned to stop worrying and love UX metrics

More Related Content

What's hot

Similar to How I learned to stop worrying and love UX metrics

More from Tammy Everts

Recently uploaded

How I learned to stop worrying and love UX metrics

Editor's Notes