Usability testing
楊政達
(1) 期末Project預告
(2) 今天: Usability Testing
期末Project
• 6/13 晚上六點開始
• 原型與使用測試報告 6/20?
– App的草創、構想、測試、發展、定案過程
等等
• 發表
–App形象影片
• 至多 1分半 ~ 2分鐘 + 5-6分鐘口頭說明
• 可以是使用介紹、App概念等等
• 廣告! 推銷! 吸引人!
• 呈現形式不拘:
– 搞笑的 Absurd Smartphone APP
– 情境的 Google Search App
– ….
Usability Testing
Usability is often an oversight
Why does usability matter?
• Usability can save time, money and lives
• On the web, usability is a precondition for
survival
• On commercial off the shelf software
applications, usability can determine
market share
Bad website example
What is usability?
• ISO 9241-11
– “the extent to which a product can be used by
specified users to achieve specified goals with
effectiveness, efficiency, and satisfaction in a
specified context of use.”
• Usability Professionals Association
– Is an approach that incorporates direct user
feedback throughout the development cycle
in order to reduce costs and create products
and tools that meet user needs
• Steve Krug,
– “..making sure that something works well: that
a person of average (or even below average)
ability and experience can use the thing --- for
it’s intended purpose without getting
hopelessly frustrated”
Benefits of Usability
• Increased productivity
• Decreased training and support costs
• Increased sales and revenues
• Reduced development time and costs
• Reduced maintenance costs
• Increased customer satisfaction
Commonality of usability
definitions
• A user is involved
• That user is doing something with a
product, system, or other thing
User-centered design (UCD)
User requirements analysis
Conceptual design…
User-centered design (UCD)
• Focus on users’ needs, tasks, and goals
• Invest in initial research and requirements
– Identify your target audience and observe
them
– Let users define product requirements
• Iterative design process
• Observe real target users using the system
What is usability testing?
• is a technique used to evaluate a product
by testing it on representative users
• test users will try to complete typical tasks
while observers watch, listen and takes
notes
Goal
• identify any usability problems
• collect quantitative data on
participants‘ performance
• determine participant's satisfaction with
the product
Can usability be measured?
• Using usability metrics
– Effectiveness – being able to complete a task
– Efficiency – amount of effort required to
complete a task
– Satisfaction – degree to which the user is
happy with his/her experience
5 E’s of usability
• Effective: How completely and accurately the work
or experience is completed or goals reached
• Efficient: How quickly this work can be completed
• Engaging: How pleasant and satisfying it is to use
• Error Tolerant: How well the product prevents
errors and can help the user recover from mistakes
• Easy to Learn: How well the product supports both
the initial and continued learning
Steps for user testing
• Plan user tests
• Conduct user tests
• Analyze findings
• Resent findings
• Modify and retest designs
Formative testing
• during the development of a product
• to mould or improve the product
• Iterative
• Output
– Usability problems and suggested fixes
– Highlight videos
Summative testing
• At the end of a development stage
• To measure or validate the usability of a
product
• "How usable is this product"
• To compare against competitor products or
usability metrics
• To generate data to support marketing claims
about usability
• Output
– Statistical measures of usability
– Reports or white papers
Benefits of usability testing
• Identify and rectify usability deficiencies
prior to product release
• Intent to create products that:
– Are useful to and valued by target audience
– Are easy to learn
– Help people to be efficient and effective
– Are satisfying (delightful) to use
Eliminating design problems
and frustration
Improving profitability
Follow the principle
• Many usability tests are worthless.
Researchers recruit the wrong kind of
participants, test the wrong kind of tasks,
put too much weight on people's opinions,
and expect participants to generate design
solutions.”
Test participants
• Participant’s background and abilities
should be representative of your product’s
intended user
• user profile – person with the relevant
behavior, skills, and knowledge who will
use your product.
How many users to test?
• 6-8 users per
test or 5 users
spread over
multiple tests
• little ROI in
testing more
than 9 users
Quantitative tests – test 20 users
Testing locations
• Lab
• Office
• bar, café
• remote testing
Usability lab
Testing in a conference room
Informal usability testing
Levels of Prototyping
Sketch / Mock-up / Final
Prototype
Usability testing process
Planning your test
• Decide what to test
– What are your objectives
– What data will you collect
• Who is your target audience?
– Write a screener
• Decide on test location
– Remote, lab, conference room, coffee shop
• Write tasks that meet your objectives
Deciding what to test
• Understand requirements
– What do users want to accomplish?
– What does the company want to accomplish?
• Determine the goals
– What tasks does the website or application
support?
• Decide on the area of focus
– Tasks that have the most impact on your site
– Typical tasks
– Most critical tasks
Test plan
• Purpose, goals, and objectives
• Participant characteristics
• Method (test design)
• Task list
• Test environment, equipment, and logistics
• Test moderator role
• Evaluation measures (data to be collected)
• Report contents and presentation
Recruiting users
• Recruit internally or outsource to agency?
• Sources of test candidates
– Your own company’s list of existing customers
– Referrals from sales and marketing
– Advertising on Craigslist
– Company’s web site or blog
– Societies and Associations
Prepare test materials
• Orientation script
• Background questionnaire
• Recording consent
• Pre-test questionnaire
• Data collection tools
• Task scenarios
• Post-test questionnaire
• Debriefing topics
Task types
• First impression questions
– What is your impression of this home page or
application?
• Exploratory task
– Open-ended / research-oriented
– e.g. Find a cellular phone plan for yourself
• Directed tasks
– Specific / answer-oriented
– e.g. Find contact information for customer
support
Metrics
• Task success
• Task time
• Errors
• Efficiency
– Number of steps required to perform a task
• Self-reported metrics
– Likert scale
– Do you prefer A or B?
– Questionnaires
Prepare the prototype
Conducting a Test
Test moderator conduct
• Put the participants at ease
• Give participants time to work through
hindrances
• Offer appropriate encouragement
• Ask non-leading questions
Thinking aloud methodology
• Observe user behavior
• Listen to user feedback
• Facilitator stays quiet, observes, take notes
• Test one user at a time
• Mainly qualitative
Debriefing
• Exploring and reviewing the participant’s
actions during the test
• Goal – understand why every error,
difficulty and omission occurred for every
participant for every session.
• Debrief with observers too.
Analyze and present
Let’s try
體驗劇場_1050524_W14_易用性測試_楊政達

體驗劇場_1050524_W14_易用性測試_楊政達

  • 1.
  • 2.
  • 3.
    期末Project • 6/13 晚上六點開始 •原型與使用測試報告 6/20? – App的草創、構想、測試、發展、定案過程 等等
  • 4.
    • 發表 –App形象影片 • 至多1分半 ~ 2分鐘 + 5-6分鐘口頭說明 • 可以是使用介紹、App概念等等 • 廣告! 推銷! 吸引人! • 呈現形式不拘: – 搞笑的 Absurd Smartphone APP – 情境的 Google Search App – ….
  • 5.
  • 6.
    Usability is oftenan oversight
  • 7.
    Why does usabilitymatter? • Usability can save time, money and lives • On the web, usability is a precondition for survival • On commercial off the shelf software applications, usability can determine market share
  • 8.
  • 10.
    What is usability? •ISO 9241-11 – “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” • Usability Professionals Association – Is an approach that incorporates direct user feedback throughout the development cycle in order to reduce costs and create products and tools that meet user needs
  • 11.
    • Steve Krug, –“..making sure that something works well: that a person of average (or even below average) ability and experience can use the thing --- for it’s intended purpose without getting hopelessly frustrated”
  • 12.
    Benefits of Usability •Increased productivity • Decreased training and support costs • Increased sales and revenues • Reduced development time and costs • Reduced maintenance costs • Increased customer satisfaction
  • 13.
    Commonality of usability definitions •A user is involved • That user is doing something with a product, system, or other thing
  • 14.
    User-centered design (UCD) Userrequirements analysis Conceptual design…
  • 15.
    User-centered design (UCD) •Focus on users’ needs, tasks, and goals • Invest in initial research and requirements – Identify your target audience and observe them – Let users define product requirements • Iterative design process • Observe real target users using the system
  • 16.
    What is usabilitytesting? • is a technique used to evaluate a product by testing it on representative users • test users will try to complete typical tasks while observers watch, listen and takes notes
  • 17.
    Goal • identify anyusability problems • collect quantitative data on participants‘ performance • determine participant's satisfaction with the product
  • 18.
    Can usability bemeasured? • Using usability metrics – Effectiveness – being able to complete a task – Efficiency – amount of effort required to complete a task – Satisfaction – degree to which the user is happy with his/her experience
  • 19.
    5 E’s ofusability • Effective: How completely and accurately the work or experience is completed or goals reached • Efficient: How quickly this work can be completed • Engaging: How pleasant and satisfying it is to use • Error Tolerant: How well the product prevents errors and can help the user recover from mistakes • Easy to Learn: How well the product supports both the initial and continued learning
  • 20.
    Steps for usertesting • Plan user tests • Conduct user tests • Analyze findings • Resent findings • Modify and retest designs
  • 21.
    Formative testing • duringthe development of a product • to mould or improve the product • Iterative • Output – Usability problems and suggested fixes – Highlight videos
  • 22.
    Summative testing • Atthe end of a development stage • To measure or validate the usability of a product • "How usable is this product" • To compare against competitor products or usability metrics • To generate data to support marketing claims about usability • Output – Statistical measures of usability – Reports or white papers
  • 23.
    Benefits of usabilitytesting • Identify and rectify usability deficiencies prior to product release • Intent to create products that: – Are useful to and valued by target audience – Are easy to learn – Help people to be efficient and effective – Are satisfying (delightful) to use
  • 24.
  • 25.
  • 26.
    Follow the principle •Many usability tests are worthless. Researchers recruit the wrong kind of participants, test the wrong kind of tasks, put too much weight on people's opinions, and expect participants to generate design solutions.”
  • 27.
    Test participants • Participant’sbackground and abilities should be representative of your product’s intended user • user profile – person with the relevant behavior, skills, and knowledge who will use your product.
  • 28.
    How many usersto test? • 6-8 users per test or 5 users spread over multiple tests • little ROI in testing more than 9 users
  • 29.
  • 30.
    Testing locations • Lab •Office • bar, café • remote testing
  • 31.
  • 32.
    Testing in aconference room
  • 33.
  • 34.
  • 35.
    Sketch / Mock-up/ Final Prototype
  • 36.
  • 37.
    Planning your test •Decide what to test – What are your objectives – What data will you collect • Who is your target audience? – Write a screener • Decide on test location – Remote, lab, conference room, coffee shop • Write tasks that meet your objectives
  • 38.
    Deciding what totest • Understand requirements – What do users want to accomplish? – What does the company want to accomplish? • Determine the goals – What tasks does the website or application support? • Decide on the area of focus – Tasks that have the most impact on your site – Typical tasks – Most critical tasks
  • 39.
    Test plan • Purpose,goals, and objectives • Participant characteristics • Method (test design) • Task list • Test environment, equipment, and logistics • Test moderator role • Evaluation measures (data to be collected) • Report contents and presentation
  • 40.
    Recruiting users • Recruitinternally or outsource to agency? • Sources of test candidates – Your own company’s list of existing customers – Referrals from sales and marketing – Advertising on Craigslist – Company’s web site or blog – Societies and Associations
  • 41.
    Prepare test materials •Orientation script • Background questionnaire • Recording consent • Pre-test questionnaire • Data collection tools • Task scenarios • Post-test questionnaire • Debriefing topics
  • 42.
    Task types • Firstimpression questions – What is your impression of this home page or application? • Exploratory task – Open-ended / research-oriented – e.g. Find a cellular phone plan for yourself • Directed tasks – Specific / answer-oriented – e.g. Find contact information for customer support
  • 43.
    Metrics • Task success •Task time • Errors • Efficiency – Number of steps required to perform a task • Self-reported metrics – Likert scale – Do you prefer A or B? – Questionnaires
  • 44.
  • 45.
  • 46.
    Test moderator conduct •Put the participants at ease • Give participants time to work through hindrances • Offer appropriate encouragement • Ask non-leading questions
  • 47.
    Thinking aloud methodology •Observe user behavior • Listen to user feedback • Facilitator stays quiet, observes, take notes • Test one user at a time • Mainly qualitative
  • 48.
    Debriefing • Exploring andreviewing the participant’s actions during the test • Goal – understand why every error, difficulty and omission occurred for every participant for every session. • Debrief with observers too.
  • 49.
  • 50.