![Designers struggle with
problem-solving [E.g., Nigel Cross]](https://image.slidesharecdn.com/part1introduction-150624063229-lva1-app6891/75/Model-Based-User-Interface-Optimization-Part-I-INTRODUCTION-At-SICSA-Summer-School-on-Computational-Interaction-2015-14-2048.jpg)
![!
!
Black-box
methods
(e.g., simulated
annealing)
Optimizer
Fitts’ lawBigrams
Objective function
literature. Our approach is directly usable in the available IP
solvers such as CPLEX and Gurobi.
BACKGROUND: THE LETTER ASSIGNMENT PROBLEM
Given n letters and n keyslots, the task of the letter assign-
ment problem is to minimize the average cost ck` of selecting
letter ` after k weighted by the bigram probability pk`:
min
X
k
X
`
pk` · ck` (1)
It is an instance of the Koopmans-Beckman Problem [15, 6],
which is NP-hard. As we will see later, this problem can be
modeled with a quadratic function on binary variables. Thus,
it belongs to a broad class of problems called quadratic as-
signment problem (QAP) where the goal is to minimize the
total pair-wise interaction cost [22, 24].
Except for a few papers using heuristic cost functions (e.g.,
[9]), the Fitts-bigram energy is used (see also [4]). Here, the
Task
n!##
Open#
Save##
Save#as...#
Close#
...#
About#
n!#
A B C D E F G H I J
K L M N O P Q R
S T U V W X Y
A
A B C D E F G H I
J K L M N O P Q R
S T U V W X Y Z
A B C D E F G H I
J K L M N O P Q R
S T U V W X Y Z
Letter assignment
Keyboard optimization
See Karrenbauer & Oulasvirta UIST 2014](https://image.slidesharecdn.com/part1introduction-150624063229-lva1-app6891/75/Model-Based-User-Interface-Optimization-Part-I-INTRODUCTION-At-SICSA-Summer-School-on-Computational-Interaction-2015-37-2048.jpg)
![ORDER NOTATION FOR
COMPUTATIONAL COMPLEXITYIntroduction Algorithms Spanning Trees
Notation for Functions f 2 . . .
!"#$%&"'(')"&%*"+%,+"-'.%"*%*/&0')"&1
!"#$ %&'()*')
!"+&,n$ +&,*-.)/0.%
!"1+&,n2c$ 3&+4+&,*-.)/0.%
&"n$ (56+.'7*-
O"n$ +.'7*-
!"n+&,n$ 85*(.+.'7*-
!"n9$ 85*:-*).%
3&+4'&0.*+
!"cn$ 7;3&'7').*+
!"n<$ =*%)&-.*+ &->%&06.'*)&-.*+
Lecture 1: sheet 26 / 43 Marc Uetz Discrete Optimization
[Marc Uetz - lecture notes on discrete optimization]
Complexity is the number of basic instructions that the algorithm performs until termination](https://image.slidesharecdn.com/part1introduction-150624063229-lva1-app6891/75/Model-Based-User-Interface-Optimization-Part-I-INTRODUCTION-At-SICSA-Summer-School-on-Computational-Interaction-2015-44-2048.jpg)
![COMPUTATION TIMESSay, we can do 2.2 · 109 operations per second (2.2 GHz)
algorithm A’s computation timea
, for tA(n) =
n log n n log n n2
n3
2n
16 ⇡0 ⇡0 ⇡0 0.002 ms 0.03 ms
64 ⇡0 ⇡0 0.002 ms 0.12 ms 266 y
256 ⇡0 0.001 ms 0.06 ms 7.6 ms 1.6 · 1060
y
4.096 ⇡0 0.02 ms 7.6 ms 31 s ???
65.536 ⇡0 0.47 ms 2 s 78 h ???
16.7 Mio ⇡0 0.2 s 35 h 68066 y ???
a
s=second, ms=1/1000 s, h=hour, y=year
Lecture 1: sheet 33 / 43 Marc Uetz Discrete Optimization
[Marc Uetz - lecture notes on discrete optimization]](https://image.slidesharecdn.com/part1introduction-150624063229-lva1-app6891/75/Model-Based-User-Interface-Optimization-Part-I-INTRODUCTION-At-SICSA-Summer-School-on-Computational-Interaction-2015-45-2048.jpg)
![Designers struggle with
problem-solving [E.g., Nigel Cross]](https://crownmelresort.com/image.slidesharecdn.com/part1introduction-150624063229-lva1-app6891/75/Model-Based-User-Interface-Optimization-Part-I-INTRODUCTION-At-SICSA-Summer-School-on-Computational-Interaction-2015-14-2048.jpg)
![!
!
Black-box
methods
(e.g., simulated
annealing)
Optimizer
Fitts’ lawBigrams
Objective function
literature. Our approach is directly usable in the available IP
solvers such as CPLEX and Gurobi.
BACKGROUND: THE LETTER ASSIGNMENT PROBLEM
Given n letters and n keyslots, the task of the letter assign-
ment problem is to minimize the average cost ck` of selecting
letter ` after k weighted by the bigram probability pk`:
min
X
k
X
`
pk` · ck` (1)
It is an instance of the Koopmans-Beckman Problem [15, 6],
which is NP-hard. As we will see later, this problem can be
modeled with a quadratic function on binary variables. Thus,
it belongs to a broad class of problems called quadratic as-
signment problem (QAP) where the goal is to minimize the
total pair-wise interaction cost [22, 24].
Except for a few papers using heuristic cost functions (e.g.,
[9]), the Fitts-bigram energy is used (see also [4]). Here, the
Task
n!##
Open#
Save##
Save#as...#
Close#
...#
About#
n!#
A B C D E F G H I J
K L M N O P Q R
S T U V W X Y
A
A B C D E F G H I
J K L M N O P Q R
S T U V W X Y Z
A B C D E F G H I
J K L M N O P Q R
S T U V W X Y Z
Letter assignment
Keyboard optimization
See Karrenbauer & Oulasvirta UIST 2014](https://crownmelresort.com/image.slidesharecdn.com/part1introduction-150624063229-lva1-app6891/75/Model-Based-User-Interface-Optimization-Part-I-INTRODUCTION-At-SICSA-Summer-School-on-Computational-Interaction-2015-37-2048.jpg)
![ORDER NOTATION FOR
COMPUTATIONAL COMPLEXITYIntroduction Algorithms Spanning Trees
Notation for Functions f 2 . . .
!"#$%&"'(')"&%*"+%,+"-'.%"*%*/&0')"&1
!"#$ %&'()*')
!"+&,n$ +&,*-.)/0.%
!"1+&,n2c$ 3&+4+&,*-.)/0.%
&"n$ (56+.'7*-
O"n$ +.'7*-
!"n+&,n$ 85*(.+.'7*-
!"n9$ 85*:-*).%
3&+4'&0.*+
!"cn$ 7;3&'7').*+
!"n<$ =*%)&-.*+ &->%&06.'*)&-.*+
Lecture 1: sheet 26 / 43 Marc Uetz Discrete Optimization
[Marc Uetz - lecture notes on discrete optimization]
Complexity is the number of basic instructions that the algorithm performs until termination](https://crownmelresort.com/image.slidesharecdn.com/part1introduction-150624063229-lva1-app6891/75/Model-Based-User-Interface-Optimization-Part-I-INTRODUCTION-At-SICSA-Summer-School-on-Computational-Interaction-2015-44-2048.jpg)
![COMPUTATION TIMESSay, we can do 2.2 · 109 operations per second (2.2 GHz)
algorithm A’s computation timea
, for tA(n) =
n log n n log n n2
n3
2n
16 ⇡0 ⇡0 ⇡0 0.002 ms 0.03 ms
64 ⇡0 ⇡0 0.002 ms 0.12 ms 266 y
256 ⇡0 0.001 ms 0.06 ms 7.6 ms 1.6 · 1060
y
4.096 ⇡0 0.02 ms 7.6 ms 31 s ???
65.536 ⇡0 0.47 ms 2 s 78 h ???
16.7 Mio ⇡0 0.2 s 35 h 68066 y ???
a
s=second, ms=1/1000 s, h=hour, y=year
Lecture 1: sheet 33 / 43 Marc Uetz Discrete Optimization
[Marc Uetz - lecture notes on discrete optimization]](https://crownmelresort.com/image.slidesharecdn.com/part1introduction-150624063229-lva1-app6891/75/Model-Based-User-Interface-Optimization-Part-I-INTRODUCTION-At-SICSA-Summer-School-on-Computational-Interaction-2015-45-2048.jpg)
Uploaded byAalto University
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summer School on Computational Interaction 2015
This document provides an overview of optimization for user interface design. It discusses how user interface design problems can be formulated as optimization tasks by defining design variables, constraints, and objective functions. Various optimization algorithms from fields like operations research, computer science, and engineering can then be applied to find optimal user interface designs. The document outlines some classic optimization problems and examples of optimization in engineering design. It also discusses model-based approaches to user interface optimization that involve representing interface design problems formally and using predictive models of user behavior together with optimization algorithms to generate improved interface designs.
More Related Content
Similar to Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summer School on Computational Interaction 2015
![HCI Research as Problem-Solving [CHI'16, presentation slides]](https://cdn.slidesharecdn.com/ss_thumbnails/hci-research-as-problem-solving-chi16-160512002517-thumbnail.jpg?width=640&height=640&fit=bounds)
![Laminated_Springs[1]. Machine design practice](https://cdn.slidesharecdn.com/ss_thumbnails/laminatedsprings1-251116120255-2a3c06fb-thumbnail.jpg?width=640&height=640&fit=bounds)
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summer School on Computational Interaction 2015
- 1. Optimization for UserInterface Design Antti Oulasvirta, Anna Feit Aalto University SICSA SUMMER SCHOOL ON COMPUTATIONAL INTERACTION GLASGOW JUNE 23, 2015
- 2.
- 3.
- 5. UI DESIGN ASOPTIMIZATION DEFINING DESIGN PROBLEMS ALGORITHMIC SOLUTIONS ADVANCEDTOPICS TODAY
- 6. SCHEDULE 09:30 Introduction 10:30 LetterAssignment 11:30 Coffee 11:45 Competition! 13:00 Lunch 13:45 Results 14:00 Integer programming 14:30 Solving real problems 15:30 Coffee 15:45 Advanced topics 16:45 Discussion 17:30 Closing
- 7. SICSA SUMMER SCHOOLON COMPUTATIONAL INTERACTION DAY 2: OPTIMIZATION INTRODUCTION
- 8.
- 9.
- 10. Evaluative studies Design heuristics Userand task analysis User-Centered Design
- 12. A hierarchical menuwith 50 items can be ordered in 10
- 13.
- 14. Designers struggle with problem-solving[E.g., Nigel Cross]
- 15.
- 16.
- 17. • Mathematics • Operationsresearch • Algorithm theory • Computational complexity theory • Engineering optimization • Artificial intelligence • Economics and management science • Software engineering A MULTI-DISCIPLINARY FIELD Rao 2009
- 18. CLASSIC PROBLEMS • Assignmentproblem • Closure problem • Constraint satisfaction problem • Cutting stock problem • Integer programming • Knapsack problem • Minimum spanning tree • Nurse scheduling problem • Traveling salesman problem • Vehicle routing problem • Vehicle rescheduling problem • Weapon target assignment problem Rao 2009
- 19. ENGINEERING OPTIMIZATION 1. Designof aircraft and aerospace structures for minimum weight 2. Finding the optimal trajectories of space vehicles 3. Design of civil engineering structures such as frames, foundations, bridges, towers, chimneys, and dams for minimum cost 4. Minimum-weight design of structures for earthquake, wind, and other types of random loading 5. Design of water resources systems for maximum benefit 6. Optimal plastic design of structures 7. Optimum design of linkages, cams, gears, machine tools, and other mechanical components 8. Selection of machining conditions in metal- cutting processes for minimum production cost 9. Design of material handling equipment, such as conveyors, trucks, and cranes, for minimum cost 10. Design of pumps, turbines, and heat transfer equipment for maximum efficiency 11. Optimum design of electrical machinery such as motors, generators, and transformers 12. Optimum design of electrical networks 13. Shortest route taken by a salesperson visiting various cities during one tour 14. Optimal production planning, controlling, and scheduling 15.Analysis of statistical data and building empirical models from experimental results to obtain the most accurate representation of the physical phenomenon 16. Optimum design of chemical processing equipment and plants 17. Design of optimum pipeline networks for process industries 18. Selection of a site for an industry 19. Planning of maintenance and replacement of equipment to reduce operating costs 20. Inventory control 21.Allocation of resources or services among several activities to maximize the benefit 22. Controlling the waiting and idle times and queueing in production lines to reduce the costs 23. Planning the best strategy to obtain maximum profit in the presence of a competitor 24. Optimum design of control systems Rao 2009
- 20. DESIGN OPTIMIZATION Ford Duratorqengine • aerospace • automotive • chemical • electronics • construction • traffic • manufacturing • logistics • telecommunications
- 21. NUMEROUS METHODS 1.2 HistoricalDevelopment 3 Table 1.1 Methods of Operations Research Mathematical programming or Stochastic process optimization techniques techniques Statistical methods Calculus methods Statistical decision theory Regression analysis Calculus of variations Markov processes Cluster analysis, pattern recognitionNonlinear programming Queueing theory Geometric programming Renewal theory Design of experiments Quadratic programming Simulation methods Discriminate analysis (factor analysis)Linear programming Reliability theory Dynamic programming Integer programming Stochastic programming Separable programming Multiobjective programming Network methods: CPM and PERT Game theory Modern or nontraditional optimization techniques Genetic algorithms Simulated annealing Ant colony optimization Particle swarm optimization Neural networks Fuzzy optimization HISTORICAL DEVELOPMENT The existence of optimization methods can be traced to the days of Newton, Lagrange, and Cauchy. The development of differential calculus methods of optimization was Rao 2009
- 22.
- 23. USER INTERFACE DESIGN Graphicaluser interfaces Consumer electronics Automotive interfaces Input devices Web user interfaces Gestural interaction Mobile interfaces Dialogue interfaces
- 24. MODERN METHODS FOR DESIGNOPTIMIZATION • Modern methods of engineering optimization • Genetic algorithms: 1975 • Ant colony optimization:1992 • Particle swarm optimization: 1995 Rao 2009
- 25. COMBINATORIAL OPTIMIZATION WITHBLACK-BOX METHODS • High dimensional problems • Exhaustive search impossible • Expensive computations • E.g., user simulations • Multiple objectives • Uncertainty, missing knowledge,changing conditions • Complex or un-analyzed objective functions
- 26.
- 27. GOALS Confidence comes withexperience 1.Ability to define UID problems as optimization tasks 2.Ability to apply heuristic methods 3.Knowledge about mathematical approaches 4.Knowledge about state-of-the-art in UI optimization
- 28. TEACHING APPROACH • Breadthplus some depth • Lectures on methods, applications, and issues • One longer exercise and several pen-and-paper exercises
- 29.
- 31. BY THIS EVENING,YOU WILL HAVE… • Knowledge allowing you to replicate basically all work until 2013 • Including CHI’12 Best paper award and our UIST’13 paper • Broader perspectives to methods, issues and possibilities in UI optimization • Skills to formulate new problems
- 32. YOU WILL BETORN • It’s easy! • Anybody can optimize with black-box methods. (The question is how well) • It’s hard! • Solving real problems require insight in problem definition, modeling, and algorithms
- 33.
- 34. WHAT IS OPTIMIZATION? •The act of obtaining the best result under given circumstances • The process of finding the conditions that give the best value ! • Minimization and maximization are equivalent Rao 2009
- 35. GENERAL FORM • Theoptimum is the design vector X with the lowest value • Constraints can be added • E.g., g(X) < 1 Find X = 8 >>< >>: x1 x2 · · · xn 9 >>= >>; which minimizes f(X) Rao 2009
- 36. COMBINATORIAL PROBLEMS P =(S, ⌦, f) Problem Finite set of candidate solutions Constraints Evaluation function
- 37. ! ! Black-box methods (e.g.,simulated annealing) Optimizer Fitts’ lawBigrams Objective function literature. Our approach is directly usable in the available IP solvers such as CPLEX and Gurobi. BACKGROUND: THE LETTER ASSIGNMENT PROBLEM Given n letters and n keyslots, the task of the letter assign- ment problem is to minimize the average cost ck` of selecting letter ` after k weighted by the bigram probability pk`: min X k X ` pk` · ck` (1) It is an instance of the Koopmans-Beckman Problem [15, 6], which is NP-hard. As we will see later, this problem can be modeled with a quadratic function on binary variables. Thus, it belongs to a broad class of problems called quadratic as- signment problem (QAP) where the goal is to minimize the total pair-wise interaction cost [22, 24]. Except for a few papers using heuristic cost functions (e.g., [9]), the Fitts-bigram energy is used (see also [4]). Here, the Task n!## Open# Save## Save#as...# Close# ...# About# n!# A B C D E F G H I J K L M N O P Q R S T U V W X Y A A B C D E F G H I J K L M N O P Q R S T U V W X Y Z A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Letter assignment Keyboard optimization See Karrenbauer & Oulasvirta UIST 2014
- 38. KEY CONCEPTS • Decisionvariables • Constraints (rules) • Objective function • types: minimization, maximization, existence, feasibility • Names: loss function, cost function, utility function, fitness function, energy function
- 39. OBJECTIVE FUNCTION • Thefollowing operations on the objective function will not change the optimum solution x*: • multiplication or division of f(x) by a positive constant c • addition or subtraction of a positive constant c to/from f(x) Rao 2009 Figure 1.1 Minimum of f (x) is same as maximum of −f (x). cf(x) cf(x) f(x) f(x) f(x) f(x) f(x) cf* f* f* x* x x* x c + f(x) c + f* Figure 1.2 Optimum solution of cf (x) or c + f (x) same as that of f (x).
- 40. DIMENSIONS OF PROBLEMS •Existence of constraints • Types of functions involved • Linear, quadratic and other nonlinear • Permissible values of design variables • Binary, integer, continuous, mixed • Deterministic vs. stochastic problems • Separability of objective function • Number of objectives Rao 2009
- 41. SEARCH CONCEPTS • Variableassignment • Constraint satisfaction • Search neighborhood • N: S —> 2S Rao 2009
- 42. SEARCH SPACE • Sizeof search space • Types of solutions • Feasible solution • Optimal solution / optimum set • Local • Global • Convergence Rao 2009
- 43.
- 44. ORDER NOTATION FOR COMPUTATIONALCOMPLEXITYIntroduction Algorithms Spanning Trees Notation for Functions f 2 . . . !"#$%&"'(')"&%*"+%,+"-'.%"*%*/&0')"&1 !"#$ %&'()*') !"+&,n$ +&,*-.)/0.% !"1+&,n2c$ 3&+4+&,*-.)/0.% &"n$ (56+.'7*- O"n$ +.'7*- !"n+&,n$ 85*(.+.'7*- !"n9$ 85*:-*).% 3&+4'&0.*+ !"cn$ 7;3&'7').*+ !"n<$ =*%)&-.*+ &->%&06.'*)&-.*+ Lecture 1: sheet 26 / 43 Marc Uetz Discrete Optimization [Marc Uetz - lecture notes on discrete optimization] Complexity is the number of basic instructions that the algorithm performs until termination
- 45. COMPUTATION TIMESSay, wecan do 2.2 · 109 operations per second (2.2 GHz) algorithm A’s computation timea , for tA(n) = n log n n log n n2 n3 2n 16 ⇡0 ⇡0 ⇡0 0.002 ms 0.03 ms 64 ⇡0 ⇡0 0.002 ms 0.12 ms 266 y 256 ⇡0 0.001 ms 0.06 ms 7.6 ms 1.6 · 1060 y 4.096 ⇡0 0.02 ms 7.6 ms 31 s ??? 65.536 ⇡0 0.47 ms 2 s 78 h ??? 16.7 Mio ⇡0 0.2 s 35 h 68066 y ??? a s=second, ms=1/1000 s, h=hour, y=year Lecture 1: sheet 33 / 43 Marc Uetz Discrete Optimization [Marc Uetz - lecture notes on discrete optimization]
- 46. EXAMPLE FROM MENUOPTIMIZATION • Convergence the best method at around 10 minutes • Although the worse method still improved, catching up would take hours Figure 5: Temporal performance of the optimizer system for the Firefox and Seashore case. In our test setup, 10,000 iterations takes about 10 minutes. Cost- About 10 minsFrom Bailly et al. UIST’13
- 47. CONSTRAINT SURFACE 3. Boundand acceptable point 4. Bound and unacceptable point All four types of points are shown in Fig. 1.4. Figure 1.4 Constraint surfaces in a hypothetical two-dimensional design space. Rao 2009
- 48. PERMUTATION MATRIX P = 2 4 10 0 0 0 1 0 1 0 3 5 Slot 1 Slot 2 Slot 3 Element 1 Element 2 Element 3 P = {1,3,2} Rao 2009
- 49. APPLICATION correct optimization algorithms,and how to combine as the situation may demand. real world problem algorithm, model, solution technique computer implementation analysis validation, sensitivity analysis numerical methods verification Rao 2009
- 50. CRITERIA: CHOOSING ANALGORITHM 1.The type of problem to be solved 2.The availability of a ready-made computer program 3.The calendar time required for the development of a program 4.The necessity of derivatives of the functions f and gj, j = 1,2,...,m 5.The available knowledge about the efficiency of the method 6.The accuracy of the solution desired 7.The programming language and quality of coding desired 8.The robustness and dependability in finding the optimum 9.The generality of the program for solving other problems 10.The ease with which the program can be used Rao 2009
- 51. SUMMARY • Basic conceptsfor single criterion combinatorial optimization • Objective function • Constraints • Global vs. local optima • Algorithm a matter of success vs. fail
- 52.
- 53. “The Design Oracle” Guaranteedquality Explicit criteria Perfect psychologist Fast, inexpensive
- 54. Design Space* Knowledge * Inreality, multidimensional
- 55. Results from astudy Design heuristics Psychophysics functions Model or simulation
- 56. MODEL-BASED INTERFACE OPTIMIZATION Designer Generate Evaluate Optimizer Model Behavioral sciences Socialsciences … Computer science Design studies
- 57. The Psychology of Human-Computer Interaction 1983 ! Cognitive simulation ! Raised operations research as a model discipline ! “Tortoise of accumulative science and the hare of intuitive design”
- 58. GOMS Limited applicability Hard touse Task definition Must be re-run for every design
- 59. Formal analysis of UIDproblems A B C D E F G H I J K L M N O P Q R S T U V W X Y A A B C D E F G H I J K L M N O P Q R S T U V W X Y Z A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Representation of problem in code Optimized UI Combinatorial optimization Predictive model of user behavior Literature, Experiments
- 60. APPLICATION PROCEDURE 1. Choosedecision variables 2. Represent the design space in code 3. Formulate the objective function 4. Obtain input values and set weights 5. Choose algorithm 6. Execute 7. Evaluate / test hypothesis
- 61. USES OF OPTIMIZATIONIN UID 1.Find the best design for a given task 2.Find the best task for a given design 3.Identify how far from the optimum a present UI is 4.Explore design space 5.Change a UI to maximally improve it 6.Find the best design for contradictory conditions
- 62. BENEFITS • Derivation ofUI designs from first principles • A problem-solving tool • Encourages exact formulation of design task • All assumptions can be scrutinized • All outcomes are testable “hypotheses”
- 63. TYPES OF MODELS •If-then heuristics (e.g., design heuristics) • Verbal theories (e.g., mental models theory) • Conceptual models (e.g., task analysis) • Statistical models (e.g., technology adoption model) • Analytical models (e.g., Fitts’ law, models) • Simulation models (e.g., cognitive models)