Understanding the Science Aspect in Interaction Design – a Summary of “Laws” in the Field

The theoretical part of HCI is very rich. There are plenty of guidelines, principles, theories and laws that constitute universal truth designers can depend upon. The first three are in general concrete and usually proposed by gurus and experienced designers, while the laws are more abstract and mostly discovered by experiments. However, many laws are misunderstood or misused by designers. In this blog article, I will be talking about many common HCI “laws/models” that can be expressed by numbers, formulas or graphs. My list of laws/models are not complete, but it includes many useful laws as follows:

  • Fitts’ Law and Accot-Zhai steering Law
  • Hick-Hyman’s Law
  • Weber’s Law, Fechner’s Law and Steven’s Law
  • Miller’s Law of Short-term Memory Load
  • Power Law of Practice
  • Pareto and Zipf Laws
  • Nielsen’s Response Time Limits Law
  • Serieal Position Effect (Hermann Ebbinghaus Law)
  • The Rule of Thirds
  • Gestalt Law and Praegnanz Law

(1) Fitts’ Law and Accot-Zhai steering Law

If you were a student in HCI, then Fitts’ law might be the first law you’ve learned in class. I’ve already talked about Fitts’ law in one of my previous blog articles. This law predicts that the time required to rapidly move to a target area is negatively proportional to the distance to the target and positively proportional to the size of the target.  This law is often used to model the pointing interaction, and is widely applied in interface design. This law can be expressed as the following formula :

  • T = a + b \log_2 \Bigg(1+\frac{2D}{W}\Bigg) (formula image is courtesy of [1])

where T is the average time taken to complete a movement when interacting the user interface,  D is the distance from the starting point to the center of the target, and W is the width of the target measured along the axis of motion. a and b are empirical constant values that are specific to the kind of pointing action.

Since both a and b are constants, and log function with base 2 is a monotone increasing function, so T is totally depend on D/W.  By transforming the law into designer’s language, it says:

  • You should design your user interface element to be large and close to where the user starts the interaction.

Of course the larger and the closer, the less time it will cost, but you have to consider other design constraints like layout and aesthetics as well as the tradeoff between increasing speed and the cost of other factors. My blog article [2] listed some examples of how Fitts’ law is used. By summarising the examples, I list the following two implications of Fitts’ law for designers:

The most frequently used user interface elements should: 

  1.  be designed both large and proximal to the users’ pointing position. (e.g. MS Word’s Menu button)
  2.  be considered to put on the border or the corner of the screen. Due to screen size constraints, at these positions, the size of the elements can be seen as infinite. (e.g. MS Windows’ start button, Mac OS’ hot corners)


There is another law called “Steering law” or “Accot-Zhai steering Law” needs to be talked about as well. The reason why I put it here is because this law is an extension of Fitts’ law.  This can be seen from the formula:

  • 413a5a0d79a65d7be823b0ba2a71eb2d (formula image courtesy of [3])

It looks pretty similar to Fitts’ formula.  Compared to Fitts’ law , which is a general measure of movement time,  steering’ law describes the time required to navigate, or steer, through a 2-dimensional tunnel. This is where the name “steer” comes from.  It was found by Accot and Zhai in their original paper. In this formula, T is also the average time to navigate through the path;  d is not a variable,  it is the differential function; C is the path parameterized by s, W(s) is the width of the path at s, and a and b are experimentally fitted constants.  Why not a constant W and why not a constant length? Because the path does not need to have constant width and length,  the tunnel may be a curvilinear shape, and this formula gives a general form. If the width and length are really constant (most of the user interface elements are like this), then the integral part can become A/W, where A denotes the length of the tunnel and W the width.

By transforming the law into designer’s language, it says:

  • You should design your tunnel user interface element (e.g. nested dropdown menu) to be wide and short.

Steering law is not widely applicable, because you don’t always have tunnel user interface element. I can give a few steering tasks to illustrate the use.

  1.  Nested menu such as dropdown menu where a users have to navigate her pointing device through the bounding box of the menu (the tunnel) to select a target.
  2.  In virtual reality application, there are many cases e.g. you drive a car on a curvy road. You cannot drive the car outside the road,  so the road is actually a tunnel interface.

In cases like this, if you want to estimate the time to evaluation your design, you can apply the steering law.

(2)Hick-Hyman’s Law

While Fitts’ law and Accot’s law is a measure of time for physical movement, Hick-Hyman’s law aims at estimating the time it takes for a person to make decision as a result of the possible choices she has; that is, increasing the number of choices will increase the decision time logarithmically. The law can be expressed by the following law:

  • hicks (formula image courtesy of [4])

where b is a empirical constance, while n is the number of choices.  There are many myths about this law, both among designers and researchers. I have discussed this law with many UX experts here [5], you can have a look at our discussions. Well, let’s list some myths.

Myth 1 : Hick’s law predicts the time a person spent to select among n items

Here I need to highlight the word “select”, so it means you select something mechanically.  For example, if a user is given a list of countries to select his/her own origin, then he/she scroll the mouse to select the right country. This is actually out of scope of Hick’s law, since Hick’s law is not for predicting the time for deciding a goal (here, the goal is clear even before seeing the interface).  Some articles about Hick’s law came to a conclusion that you cannot show too many options for such selection tasks, otherwise it may reduce users’ reaction time. This is totally wrong, in these cases, users already know what to select, what they do is a simple scanning task to find the right item and click. It has something to do with Fitts’ law, however, since the longer the list, the longer time it takes you to reach the item.

Myth 2: Hick’s law predicts the time a person spent on making a decision among n arbitrary items

This myth depends on how you interpret it.  Hick’s original experiment was that a user performed motor tasks for selecting which of the random light was lighted up. All lights have the same probability to be selected, a user must first saw which one was lighted up, and reacted immediately, like a Whac-A-Mole game. This was the original experimental setting of Hick. In this case, we could say it is within the reach of Hick’s law.  But this applies only to simple decisions, complex decisions may not work.

Regarding applying Hick’s law in HCI, there has been much discussion since the 20 years ago. [6] gave a pretty much full story. The conclusion is that Hick’s law is not popularly used in HCI design, unlike its sister Fitt’s law. One reason is that Hick’s law mainly targets simple artificial tasks (such as pressing a button), but real-world UI tasks are never simple. There are much cognitive and motor processes involved.  Olsen and Nielsen tried to make a general conclusion about their experiments in the use of spreadsheet software that “The more possible alternative responses, the longer it takes to choose any one”. But this message can not be taken easily.  For example, when designing menus, you need to consider both the breath and depth of it. Reducing the number of items for a specific menu may also increase the depth of it, by hierarchical nesting menus. This is not always a good UI design process. However, according to Hick’s law’s logarithmic function, you double the number of menu items will only result in an incremental, but not double the reaction time. So, sometimes breadth of information visualisation might be more favourable. It totally depends on the context, there is no one-for-all solution.  Anyway, in HCI Fitts’ law is much more intuitive and effective than Hick’s law.

(3)Weber’s Law and Fechner’s Law

Weber’s law states that the ratio of the increment threshold to the background intensity is a constant. So when you are in a noisy environment you must shout to be heard while a whisper works in a quite room. And when you measure increment thresholds on various intensity backgrounds, the thresholds increase in proportion to the background. This law can be illustrated with the following equation.


Actually Webber’s law is also called Weber-Fechner’s law, the German psychologist Fechner proposed another similar law with two parts. The first part is that two stimuli will be discrinimable if they generate a visual response that exceeds some threshold. The second part is that the visual response R to an intensity I is given by the equation.

Fechner’s law provides an explanation for Weber’s law. Fechner’s explanation has two parts. The first part is that two stimuli will be discriminable if they generate a visual response that exceeds some threshold. The second part is that the visual response R to an intensity I is given by the equation R = log(I ). This relation is plotted in the next graph. Suppose we call the change in response necessary for discrimination “one”. Then by Fechner’s law the two intensities indicated by arrows will be just discriminable. The discrimination threshold ∆I is given by the distance on the x-axis between I and I+∆I.

Screen Shot 2013-11-03 at 16.24.09

Web’s law and Fechners’ law are mathematically identical. The laws have revealed the relationship between the absolute stimuli increase and the perceived increment. It means, in order to achieve a perceived change, the change must be relative to the current magnitude. In user interface design, Weber’s Law can be used for  various sensory modalities in GUIs such as brightness, loudness, line length, visual weight of fonts in typography, color matching etc. Many times large amount of information is required to be displayed on a limited size computer screen. Information is displayed in various forms viz. text, pictures, drawings, maps, graphs, videos etc. Poor visual design of user interface lacks the ability of differentiating between two close enough visual stimuli e.g. two lines with different thicknesses in a map (for wide roads and narrow lanes).What is  this  threshold  of  line thickness that  may  lead  to  noticeable difference  is governed  by Weber’s law.The threshold of noticeable difference between color shades is also governed by Weber’s law[8].

(4)Miller’s Law of Short-term Memory Load

Observation that an individual normally can retain or process only seven give or take two (7±2) items (chunks) of information in their correct serial-order, in his or her short-term (15 to 30 seconds duration) or working memory. Reported by the Princeton University’s cognitive psychologist George A. Miller in 1956, it is accepted more as a rule of thumb  than a scientifically proven fact. Also called Miller’s Magic Number.

Many people, including HCI professionals often mistakenly use this law to denote the maximum number of items that should be shown in a menu or in a slide. This is wrong, because the items in a menu or on a slide do not require people to hold in memory, instead, they are rather persistent. I don’t really see a real usage of this law in HCI.  I remember having seen an expert from interaction-design.org wrote a blog article about this law. He said an example of applying this law in interaction design is in mainly for usedl when specific information must be memorized for later use. [9]

E-learning applications should make liberal use of chunking to aid in end-user memorization.  Chunking is also ideal in environments where an interface must compete against other stimuli for the attention or working memory of the end user (car navigation systems, cell phone, public kiosks).  Consider a health practitioner in an emergency room scenario.  They are often:

  1. Barraged by a multitude of visual and auditory stimuli such as telephones ringing, people talking, and rapid movement. 
  2. May have only moments to look at an interface to extract and memorize the important information.   
  3. May need to enter this information into disparate systems without the benefit of referring back to the source.   
  4. Use legacy systems that cannot be reprogrammed to  repopulate or carry over data from one screen to the next
  5. Are not allowed to write down information due to privacy legislation.

In the above environment, effective use of chunking can improve the usability and effectiveness of an information system.  In the table below, the left hand column (Column A) does not use chunking while the right hand column (Column B) does.  Of the two scenarios, Column B makes it much easier and faster for a health practitioner to focus on and memorize the Patient ID, especially when faced with the sensory overload of an emergency room.

Column A – Without Chunking Column B – With Chunking
Patient ID:678290234
Name: Joe Smith
Patient ID   6782  9023  4
Name: Joe Smith
DOB 02 / 11 / 1973

Well, I have to say that this scenario somewhat makes sense, but it is still not common. The education of interaction design has exaggerated the importance of Miller’s law in design, but it is not the case. Neverthelss, we must note that this law can only be used when we really need working memeory to memorize.

(5)Power Law of Practice

The Power Law of Practice is the law of learning curve. The power law of practice states that the logarithm of the reaction time for a particular task decreases linearly with the logarithm of the number of practice trials taken. It is an example of the learning curve effect on performance. It was first proposed as a psychological law by Newell & Rosenblom[10].  For example, if you know absolutely nothing about a topic, you can learn 50% of the information quickly. But when you have 50% less to learn, it takes more time to learn that final 50%.

This law is not used for guiding interface design, but it is rather a knowledge we must learn for interaction designers : a quantified measure of how our users can learn to use the user interface.

(6) Zipf’s Laws (Law of Least Effort) ad Pareto Principle

Zipf’s law refers to the fact that many types of data studied in the physical and social sciences can be approximated with a Zipfian distribution. It described phenomena where certain types of events are quite frequent, whereas other types of events are rare. In English, for instance, short words such as articles are quite frequent, whereas long words are quite rare. In music, consonant harmonic intervals are more frequent, whereas dissonant harmonic intervals are quite rare, among other examples [11]. Zipf’s law is expressed in terms of the frequency of occurance (quantity) of events, as follows: F ~ r^(-a), where F is the frequency of occurance of an event within a phenomenon, r is its statistical rank (position in an ordered list), and a is close to 1. Another formulation of Zipf’s law is: P(f) ~ 1/f^(n), where P(f) denotes the probability of an event of rank f and n is close to 1.

This law is not usually not directly using in design, but is something that helps us understand the user’s behaviour. For example, with a menu design, we might learn that the “File” menu is the most used item, comes before “Insert”, then we can estimate that the number of usage of the “Insert” item is half the frequency of “File”, etc.

However, compared to Zipf’s law, Pareto’s principle also reflects similar findings. Pareto Principle is also known as the 80/20 rule, which states that 80% of results come from 20% of actions. Named after Vilfredo Pareto who in 1906 observed that 80% of Italy’s land was owned by 20% of the population. It has become a guideline for businesses and can easily be applied to web design. Using this as a guide you can make assumptions towards your design, mainly in regards to content. If you consider that you have a bulky, bloated, web site with 50 pages. You’ve taken care and spent all this time crafting the pages to present the stacks of content as nicely as possible to the user. But of those 50 pages, 10 of them are doing 80% of the work and the other 40 pages are only doing 20%. This could lead to removing the pages, features or content that isn’t being accessed.

(7)Nielsen’s Response Time Limits Law

This law corresponds to human response times. The famous Usability guru Jakob Nielsen wrote a article about 3 response limit in user interface design.

  • 0.1 seconds gives the feeling of instantaneous response — that is, the outcome feels like it was caused by the user, not the computer. This level of responsiveness is essential to support the feeling of direct manipulation (direct manipulation is one of the key GUI techniques to increase user engagement and control.
  • 1 second keeps the user’s flow of thought seamless. Users can sense a delay, and thus know the computer is generating the outcome, but they still feel in control of the overall experience and that they’re moving freely rather than waiting on the computer. This degree of responsiveness is needed for good navigation
  • 10 seconds keeps the user’s attention. From 1–10 seconds, users definitely feel at the mercy of the computer and wish it was faster, but they can handle it. After 10 seconds, they start thinking about other things, making it harder to get their brains back on track once the computer finally does respond.

A 10-second delay will often make users leave a site immediately. And even if they stay, it’s harder for them to understand what’s going on, making it less likely that they’ll succeed in any difficult tasks. Even a few seconds’ delay is enough to create an unpleasant user experience. Users are no longer in control, and they’re consciously annoyed by having to wait for the computer. Thus, with repeated short delays, users will give up unless they’re extremely committed to completing the task[13].

An example use of these laws is Netbeans, where the UI designers applied the UI responsiveness guideline [12].

  1. 0.1 second - navigation and edit actions (e.g. folder expansion, paste in editor, navigation in editor), and painting of all menu bars must finish within this limit
  2. 1.0 second - all window and dialog openings must finish within this limit
  3. 10 seconds - all actions which are finished later than after 1 second and usually take less than 10 seconds must show some sort of busy indication (e.g. hour glass cursor or “please wait…” text); all actions taking longer than this limit are required to provide progress bar using Progress  APIs

(8)Serial Position Effect (Hermann Ebbinghaus Law)

The term serial position effect  was coined by Hermann Ebbinghaus which refers to the finding that recall accuracy of  an  item  from a list varies as a function of  its position within  the list. People tend to recall items at  the end  of  list more  reliably (the recency effect). Also the first few items are recalled more frequently than the middle items (the primacy effect).

Refer to the  list below  and  try  to  remember  the  items  in  the list.

1 Amul  Butter
2 Pen  Pencil
3 Diary Milk
4 Orange  Juice
5 Key  Board
6 Drum Sticks
7 Paper Cup

It  will  be  observed  that  you  will  more reliably recall  items in position 1, 2,6 and 7  than those in the position 3,4 and 5.

This effect can be used in user interface design. The more important or more frequently  used  items or  controls  should be placed in the beginning or end of the list. Items less frequently used should be placed in the middle of the list. A list of written words may be more easy to recall than a set of colors or symbols. In case of a list using colors the serial position effect may not be immediately seen. Here recall depends on various properties of color such as brightness, hue and saturation too. Thus a brighter color regardless of its position in the list is more likely to be recalled. This can be used to over-rule the serial position effect whenever the context demands. Thus color can be used as a design element to aid recall. For example, while designing a tool bar, an important command  which cannot  be accommodated  at  the starting or ending of a list, could be placed in middle part with a slightly brighter color, so that it still receives the attention of the user and the user tends to remember it easily and for a longer time.

(9)The Rule of Thirds

The rule of thirds is a guideline which can be applied to a huge range of things relating to composing of visual images, such as paintings and photographs. The guideline suggests that you take an image and divide it into nine equal parts. This will give you two vertical and two horizontal lines [14]. Visual elements should be placed on the lines, or the intersections between the lines as these are the areas that the user’s eyes naturally focus on. This rule is widely used in photography [14] and visual design. The take away message is that you should place the object of focus at the intersection points. See

(10)Gestalt Law and Praegnanz Law

Gestalt law is the fundamental law of human perception. It is widely used in user interface design. Its principle maintains that the human eye sees objects in their entirety before perceiving their individual parts, suggesting the whole is greater than the sum of its parts. Gestalt psychology tries to understand the laws of our ability to acquire and maintain stable percepts in a noisy world[16]. Gestalt law is composed of many sub-laws. I summarise them as CFPCSS laws:

  • Closure
  • Figure & Ground
  • Proximity
  • Continuation
  • Similarity
  • Symmetry

Gestalt law needs a full article to explain, which I will do later. In this article, I just presented the name of the laws of Gestalt theory.


[1] http://en.wikipedia.org/wiki/Fitts’s_law



[4] http://en.wikipedia.org/wiki/Hick’s_law


[6] Steven C. Seow, Information Theoretic Models of HCI: A Comparison of the Hick-Hyman Law and Fitts’ Law, 2005

[7] Olsen and Nielsen, Analysis of the Cognition Involved in Spreadsheet Software Interaction, 1988

[8] : http://iitg.vlab.co.in/?sub=72&brch=170&sim=750&cnt=1


[10]: http://en.wikipedia.org/wiki/Power_law_of_practice

[11]:Manaris, B., Machado, P., McCauley, C., Romero, J., & Krehbiel, D. (2005). Developing fitness functions for pleasant music: Zipf’s law and interactive evolution systems. In Applications of Evolutionary Computing (pp. 498-507). Springer Berlin Heidelberg.




[15]: http://upload.wikimedia.org/wikipedia/commons/f/f0/RuleOfThirds-SideBySide.gif


Some interchangeable terms in statistics

The science part of HCI lies mostly in statistics. HCI researchers are using statistical tools for testing, e.g. the effect of a new interface, interaction technique etc. I am not a statistician, I learned it partly from a user study course at the university, and partly from books and numerous free lecture slides across the web. However, I soon discovered some of the equivalent terms that are used in different tutorials, books and articles.  Let me speak about them in this blog.

1. Independent Variables (IV) and Dependent Variables (DV) 

When I started learning the basics of statistical testing couple of years ago,  IV and DV are among the first terms that I learnt. Briefly speaking, the former one can be seen as the cause, and the latter is the effect/outcome. I am not going to talk about details, since this is not the focus of this short article. I will be speaking about what other terms I’ve found in other literatures.

IV and DV are also referred to as the following:

(1) Predictor variable and Response variable

(2) Factor (Covariate) and Measure

(3) Explanatory variable and Outcome variable

(4) Regressor and Regressand (in regression analysis)

(5) Controlled variable and Measured variable

(6) Manipulated variable and Responding variable

I want to note something about (2) and (5):

Although “Controlled variable” is sometimes refers to the same thing as IV by some authors. In experiment design, it usually means something that is constant and unchanged in an experiment. We know it will affect the the experiment, but we control it to be constant across the experiment. This is similar to what some authors referred to as covariate.  A covariate is usually not manipulated, such as gender, age, temperature, walking speed of a subject. We usually also control the covariate. A factor mainly refers to categorical variables with several levels.

Anyway, all these terms are quite common in literature, and we need to be familiar with different terminologies.

2. The type of data

Regarding the type of data, there are also different terms:

(1) Nominal variable, categorical variable

Here people use the two terms interchangeably.  When a categorical variable has only two levels, then it is called dichotomous variable.

(2) Ordinal variable

This refers to e.g. questionnaire data

(3) Continuous variable, scalar variable

They refer to the same thing, but there are two sub-types: interval (for data without absolute zero) and ratio data.

3. Mixed data 

There are also several terms for mixed data, or say, mixed effect model:

(1) Multilevel model

(2) Hierarchical linear model

(3) Random coefficient model

(4) Variance component model

(5) Nested model

(6) Mixed model

(7) Random effect model

(8) Random parameter model

(9) Split-plot design

This one is pretty complex, but you’ll find all these terms quite often. Also, if you are talking about longitudinal data, someone also refers to panel data.

In short, this article is a short summary of some equivalent statistical terms.


Final Remark

Perhaps a hierarchical presentations of types of variables is needed in addition to the discussion before, and this presentation is taken from Andy Field’s book <Discovering Statistics Using SPSS>

Categorical variable:

  • Binary variable : There are only two categories
  • Nominal variable: There are more than two categories (e.g. whether someone is an omnivore, vegetarian, vegan or fruitarian)
  • Ordinal variable: The same as a nominal variable but the categories have a logical order (e.g. Likert-scale)

Continuous l variable:

  • Interval variable : equal intervals on the variable represent equal differences in the property being measured (e.g. the difference between 6 and 8 is equivalent to the difference between 13 and 15)
  • Ratio variable : the same as an interval variable, but the ratios of scores on the scale must also make sense (e.g. a score of 16 on an anxiety scale means that the person is, in reality, twice as anxious as someone scoring 8)

Uncover User Experience and Usability

The field of Interaction Design is relatively new compared to other well-established fields like maths or physics. Although quantitative research in the field has made it more science-like, we can hardly say it is pure science. Experiments in science should be reproducible and theorems in science can be strictly proved, but this is hardly realised in user research.

IxD – An unmatured field

Another manifestation of an new or un-matured field is that people use different terms to refer to the same thing.  In the field of interaction design, people actually speak different dialects of jargons. For example, some people describe what they do as “Interaction Design”, some as “Human Computer Interaction”, some may simple say “User Interface Design”.  Well, the last one might be a little bit out of date, since User Interface Design traditionally refer to graphical design.  For HCI and IxD, I would say both are interdisciplinary, are umbrella terms and both mean that we design interactive artefacts to support people communicate and interaction in their daily life.  Of course, IxD might be a bit larger than HCI, since HCI mainly focus on the design, implementation and evaluation of computing artefacts, while IxD may cover a wider range of products, without limiting itself to computing devices. Actually such examples spread everywhere in our field, and I am not going to talk about all of them.  I want to pick up two terms “user experience” and “usability”.

Usability and User Experience

If you were asked to give the aim of Interaction Design (or whatever you call it). You might answer “to enhance positive user experience and reduce negative user experience” or simply “to ensure quality user experience”. You may also answer “to make a product more usable” or “improve its usability”. In fact, each answer is correct in its own right. However, what is user experience? what is usability? Are they referring to the same thing? Does good usability ensure good user experience?

Usability is an older term compared to user experience. It basically means the ease of use of a product. To ensure usability, your system needs to satisfy a set of objective usability goals, which I will be talking about later on. Compared to usability , user experience is more subjetive. Every product that requires user interaction has user experience. It reflects the feelings a user has when use the product. These feelings include the overall feelings of the look-and-feel, engagement etc, they also include feelings when a user e.g. press a button, choose a menu or whatever they interact with the product. They might be happy, astonished or frustrated. All these contributes to the user experience of a product. Of course one cannot design the user experience, because you cannot design how a user would feel your system. One can only design for the user experience, i.e. design the user interface to trigger users’ positive feelings.

The Goals

Whenever we design a user interface by following interaction design methodologies, we must fulfil a set of usability goals and another set of user experience goals. We cannot say usability goals and user experience goals are mutually exclusive, we would rather say that they have different focus.

(1)Usability Goals

What are the usability goals? As I mentioned at the beginning of this article, the field of interaction design is still unmatured, so you will find many principles or theories that are guru-oriented, by which I mean, some pioneers in the field such as Jokob Nielsen, Ben Schneidermann, Donald Norman etc. have their own theories. Although those theories are quite similar to each other, they are not completely the same. I think I am going to write another blog article talking about HCI principles and theories, where you will find a lot of alternative principles proposed by different gurus. The difference between principles and goals is that principles are more focused on “how”, while goals are more focus on “what”. Today I just want to focus on usability goals.

Jokob Nielsen, a Danish usability consultant, is a big name in usability. He defined five components as goals for usability. I call them the LEMES goals, which are five words with L, E, M, E, S as their initials respectively. These words are kinda abstract, so usually interaction designers use these goals by formulating questions for each term. The good thing of these goals is that all dimensions are measurable.

1. Learnability (measured by time to learn)

e.g. How easy is it to learn how to use the product?

2. Efficiency (measured by time/speed to complete the task)

e.g. How efficiently can the users perform their tasks or get things done correctly?

3. Memorability (measured by retention rate)

e.g. Once you learn how to use a product, how easy is it to remember the workflow after not using it for a long while?

4.Errors (measured by error rates)

e.g.  How effective can your product prevent users from errors? For example, sometimes a door is designed in a way that it automatically locks when you close it. This is designed with this goal in mind. Of course you can argue that it may cause problems if you happen to forget your keys inside the room. So it actually prevents one error (forget to lock the door), but increase the chance of another error (forget the key at home). Another example is a water boiler, usually such a boiler can be switched off automatically when the water is boiled, this is to prevent you from forgetting to do so.

5. Satisfaction (measured by satisfaction scales)

e.g. How satisfied are the users after using the product?

Yvonne Rogers added three extra goals in her book “Interaction Design”, which are safety, effectiveness and utility. She also removed satisfaction and errors. I understand why she removed satisfaction from usability goals. Satisfaction is subjective, and hard to measure it should be classified into user experience goals. She referred safety as not only protecting users from danger in life, but also from danger in unintended behaviours of the system (errors). So this safety term actually overlaps with Nielsen’s Error goal. By effectiveness, she meant a product should not only be efficient in executing jobs, but also effective in result. For the utility goal, actually she meant a product should provide the right set of functionalities for users to perform the tasks they want.

I think Rogers’ goals and Nielsen’s goals are complementary. Of course one can propose his own goals for usability, but these 8 terms are already useful enough. As I said, these usability goals do not tell you how to ensure usability of a product, it simply tells you that you need to keep these questions in mind when designing a new product, and ask yourself repeatedly if these goals are fulfilled during the design process.

(2)Use Experience Goals

What are the use experience goals? Briefly speaking, they are a set of subjective qualities that reflects how users feel a product, and our goal is to reduce the negative feelings and enhance the positive feelings. According to Yvonne Rogers, some quality aspects are listed as follows (not complete):

Desirable aspects:

Satisfying, Helpful, Fun, Enjoyable, Motivating, Surprising, Emotionally fulfilling

Undesirable aspects:

Boring, unpleasant, frustrating, annoying

To measure these feelings of users, one can do Shadowing (a user study method) to observe how users use your product, one can also design  questionnaires to ask people’s feeling with the above listed terms with Likert-scales (emotions in opposite direction), such as “Describe your feeling after using the system : Unpleasant  1  2  3  4  5  6  7  Pleasant“.  The former method allows you to observe their instant emotions when they are using a product, but it is NOT always feasible and sometimes even impossible. For example, if you want to observe people’s emotion of listening to music while taking a shower. You cannot follow him/her in the bathroom. The latter method allows you to assess the overall feelings of users after using your product. These feelings might not be precise, but much easier to acquire.

Just as the example I gave in the previous paragraph, listening to music while taking shower may be pleasurable. Listening to music while working may be motivated for users. So if you are designing a product, depending on the context of usage, it will result in different user experience. That’s to say, user experience goals are very subjective, not like usability goals, which are just a set of objetive criteria.

Which is more important?

After presenting the differences between usability goals and user experience goals, you may wonder, which are more important when designing user interfaces? My answer goes for the latter ones. Of course,  sometimes usability and user experience are overlapping. You ensure one thing, and you inherently  have the other one as well. But there are occasions where this does not hold. For example in computer gaming, studies have shown that people love challenges. That being said, you as a designer, can sacrifice usability to make the game harder to control, but this may achieve higher user experience.

In the gaming example, I already showed low usability can achieve high user experience. However, maybe I need to talk about something else, because gaming is  kind of “outliers” in interaction design. So, next I am going to talk about recommendation systems.I just want to show why I think user experience can be more important. In many online shopping websites, including Ebay, Taobao, Amazon, also airline websites, they often recommend you products that you may potentially have interests in. Last week I bought an air ticket from easyjet.com for my summer vacation. Before I could enter the payment page, the website recommended me hotels and car rental options as shown in the following picture. (I don’t want to tell you where I am going for my vacation, so I faked the following image by choosing London as my destination) .My question is:  why? Why did the airline company offer me options for hotel booking and car rental? Which aforementioned usability goals were the designer trying to achieve?

Screen Shot 2013-06-26 at 10.03.40 PM

My answer is : None.

Easyjet provided me with these options for no usability reasons, actually it has violated one of usability goals – efficiency, because it reduced the efficiency to take me directly to the payment page.  Clearly, this design is for user experience goals. The company is trying to figure out my needs, since normally people need hotels and cars when traveling abroad. I don’t feel annoyed by these, actually I am quite happy to see these options, because I can compare the rates with booking.com.  The same reason holds for Ebay recommends you products you may like, it tries to increase your user experience, say, for example,“Surprising”. If you remove these options, we cannot say that the user experience is degraded, but with these features, users’ experience may be enhanced, without losing anything. They might even surprisingly find something interesting.  Last but not least, any recommendations can not be abused. If you overly use recommendations, the users may be annoyed.


The term user experience is becoming more and more popular. Now many companies have job positions called “user experience designer” or “user experience researcher”, because they know what really makes a difference is user experience. My conclusion is that, when designing a product, one should keep in mind both usability and user experience goals.  Usability goals are  objective criteria and are often task-oriented, which are relatively easier to evaluate. They aim at enabling users to perform tasks in a more efficient and effective way.  Good usability ensures users to have a clear mental model of the interface that enables them to confidently predict what will happen in response to their actions. In the best cases, the interface almost disappears, enabling users to concentrate on their work, exploration, or pleasure. This kind of calming environment gives users the feeling that they are “in the flow”, operating at their peak, while attaining their goals (Designing the User Interface, Ben Schneidermann).

User experience goals are more subjective, users sometimes do not know what they want, or what their tasks are. So as an interaction designer, one must understand deeply the users, including the usage context of a product, the behaviour patterns as well as potential needs. UX designers have to think about desirability, about how to frame the offering to increase the usage. If possible, we also need to try our best to influence/persuade users.

ManyKeyboard – Using Java HID API to handle Multiple Keyboard Input

I want to announce a small Java library I made for handling multiple keyboard input on a single computer. I’ll start by giving some background knowledge.

Three years ago, at our lab we had the need to write a Single Display Groupware (SDG). Let us temporarily forget about Multitouch tabletop and only focus on traditional mouse and keyboard.

What if multiple mice and keyboard are connected to the same computer. For historical reasons, contemporary operating systems natively support only one mouse and keyboard. I am not saying you cannot connect more mice and keyboard to your computer. Of course you can, but the OS treat them in the same way. For SDG based on mice and keyboards, one problem we are facing is how to discriminate one mouse/keyboard from another.

Let’s talk about mouse and keyboard separately in the following sections.

1. How to handle Multiple mice?

At that time, we did not think of implementing our own solution. We found the “SDG Toolkit” (http://grouplab.cpsc.ucalgary.ca/cookbook/index.php/Toolkits/SDGToolkit) from Saul Greenberg’s lab at the University of Calgary. Saul Greenberg is a big name in HCI. Some years ago in their group they did research in SDG, so they developed the toolkit. The library was quite old and only supports Microsoft .NET 2.0. Later on, I also found the Microsoft Multipoint SDK (http://www.microsoft.com/multipoint/mouse-sdk/) which is also capable of handling multiple-mice. Of course, here I mean handling multiple-mice within the application level, not at the OS level. Both solutions are only for Windows. In order to have it for other platforms, there is another C library called ManyMouse (https://icculus.org/manymouse/) which works on all three major OS’s. There is also a Java port of it.  The Java-based Multitouch Framework Mt4j also uses ManyMouse for simulating multiple touch.

Until now, you see that handling multiple mice input seems to have many alternative solutions. Then, how about keyboards?

2. How to handle multiple keyboards

SDG toolkit actually also support multiple keyboard for Windows. In Windows, you can of course make your own library by following the approach described in this article : http://www.codeproject.com/Articles/17123/Using-Raw-Input-from-C-to-handle-multiple-keyboard .

If you are looking for something for another OS, e.g. in Java. Unfortunately I did not find an existing solution. Of course, you are not possible to find a pure Java solution, because this involves handling low-level input.

I was actually inspired by the article I mentioned previously. I think it should not be difficult to make my own library if I am able to get the low level input of the operating system. I discovered HID API (http://www.signal11.us/oss/hidapi/) which allows an application to interface with USB and Bluetooth HID-Class devices on any OS platform. There is also a Java port of this library called javahidapi (https://code.google.com/p/javahidapi/).

I then started building my own library which is named “ManyKeyboard” with this Java api. It is not difficult. I made the source code available in my BitBucket repository  (https://bitbucket.org/nanli/manykeyboard/src). If you are interested, you can have a try, there is a TestManyKeyboard.java file. Just run it.

What I actually wrote make this work is the following:

1. Read HID specifications and find the usage page of Keyboard device (usage page = 6)

2. Connect to all detected keyboard (with usage page =6 ).

3. Get the input report from the keyboard. An input report is the binary data of your USB device. For keyboard, you can get which key is pressed and released.

4. Fire the corresponding event.

In order to use this library, you can just create a class which extends ManyKeyEvent Listener, and initlalize the library in the following sequence.


For complete source code, please see my bitbucket link given above. There is an example application in my Bitbucket repository.


I created a Java lib called ManyKeyboard to handle multiple keyboard input. From now on, in order to develop an SDG application, you can do it with Java on any platform. Suppose you have a multitouch tabletop application and you want to use physical keyboard instead of onscreen keyboard, considering its lack of tactile feedback. Here is the solution.

UI design and Fitts Law

Fitts’ law is one of the most important law for aiding user interface design. It provides a tool for predicting the time required to reach a target, given the size of the target and the distance to it.

Basically it describes the bigger and closer a target is, the faster you can grab it. That being said, the time required to reach a target is proportional to the size of the target, and inversely proportional to the distance to the target. Fitts law is widely applied in user interface design.

For example, the Ribbon interface office button of Microsoft Word is designed to be large, so as to be reached easily.


The Chrome tab is located at the top, making it easy for a user to choose a tab (Here, the distance to the tab can be seen as infinity, because your mouse can not move beyond the border of the screen). Unfortunately this only applies to Windows, because on a Mac, you have the menu bar on top. So Chrome actually leaves some blank space between the tab and the menu bar, so that users would not click on the menu bar accidentally, while their real aim is to switch a tab. Although Mac OS is generally speaking user friendly, and the menu bar does provide unified user experience, but I the sacrifice on Fitt’s law really make me very sad.


Another case where Mac OS does not apply Fitt’s law is the close/minimize button for each window. The buttons are designed to be very small and round, making them very hard to click.  I don’t know why Apple never change the way the present the close/minimize buttons, maybe due to historical reasons. But this again makes me very disappointed.


Actually, Mac OS is not always forgetting about Fitt’s law. For example the dock, and the Mission control and hot corners (infinite distance) are all good examples of appying Fitt’s law to UI design.

dock-de-Mac Lion_Mission_Control

Similar to the close/minimize button on Mac OS, it seems that Apple bring the violation of Fitts law to iOS as well. Whenever you want to delete or close an app on an iOS device. You often see these small round buttons on the top-left corners.


You have to click the small button with your finger in order to close/delete an App. This really gives frustrating user experience to me. Sometimes I need to tap several times to hit the button.  I often ask. If Apple thinks 4inch is the perfect size of a mobile phone for human’s hand to hold and manipulate, if they think the thumb-sized application icons are perfect to hit with thumb. Well, all these are kind of true due to ergonomics. Why do they think the small close button can be tapped effortless?

Fortunately, Apple just release iOS 7 last night. My colleague installed it on his device and showed it to me. I find the this Jonathan Ive designed system really fixed the problem. Now you can kill an application by swipe it with your finger.


(GIF Source here: http://www.cultofmac.com/231322/how-to-close-apps-in-ios-7-gif/)

Truly speaking, similar features are already available for Android devices and Jailbreaked iPhones. But now Apple make it officially. This is an “exciting news”!  It is actually quite difficult for Apple to admit something they designed are flawed, but this time. Jonthan did a good job.

Anyway, iOS and Mac OS are far better than Windows and Android in terms of user experience. But they can still can do even better, I believe.

Paper Review: Anatomy of the Unsought Finding. Serendipity: Origin, History, Domains, Traditions, Appearances, Patterns and Programmability

Today I read a paper which was published in 1994, titled “Anatomy of the Unsought Finding. Serendipity: Origin, History, Domains, Traditions, Appearances, Patterns and Programmability” by Pek Van Andel, a Dutch researcher. The reason why I read this paper is because I want to deeply understand the term serendipity, which is part of my research theoretical foundations. Now I am going to review and summarize this paper.

The contribution of this paper is as follows:

1. Definition of serendipity: the art of making an unsought finding.

2. Origin of the word: Jan. 28 1754 by Horace Walpole. He read a persian fairy tale called “The Three Princes of Serendip” — three princes were always making discoveries by accident and sagacity of things that they were not in quest of.  And he called this phenomenon “serendipity”.

3. Actually this paper is mainly about serendipity of discovery in four domains:

(1) Science (discover something new and true) (2)Technology (new and useful ) (3) Art (new and facinating) (4)Daily life . However, how true/ usefu/fascinating can not be proved in one day, maybe in the future.

4. The author discribminate three appearances of serendipity.

(1) Positive serendipity: A surprising fact or relation is seen and followed by a correct abduction, for example, the discovery of X-ray

(2) Negative serendipity: A surprising fact or relation is seen but not investigated by the discoverer, Columbus’s New Word’ for example.

(3) Pseudoserendipity: To discover, invent, create something you were looking for in a surprising way, Fleming discovered penicillin after he had discovered lysozyme.

The author also identify three kinds of reasoning. Deduction, induction and retroduction (abduction). Deduction proves that something must be; Induction shows that something actually is operative; Abduction merely suggests that something may be. Its only justification is that from its suggestion deduction can draw a prediction which can be tested by induction, and that, if we are ever to learn anything or to understand phenomena at all, it must be by abduction that this is to be brought about.

5. Patterns of serendipity

The author summarized 17 patterns of serendipity. I will not list all of them, just a few: Analogy, one suprising observation, repetition of a surprising observation, successful error, etc.

6. In practice, when serendipity plays a role, this role is normally secondary (supporting) but essential. Several studies indicate that commercially successful innovations for example are for about eighty percent answers to an already pre-existing and known problem, like the “pill”, but in the remaining twenty percent, it appears that something was discovered before there was a demand for it, like X-rays.

7. Can we programme serendipity? Serendipitious luck may come unexpectedly, but it does so only in a mind “prepared” by previous interest, thought and/or experience, as in the examples of pseudoserendipity, e.g. Fleming. The very moment we can programme is, that, if the unforeseen happens, the system alerts the user and incites him to observeand act by himself by trying to make a correct abduction of the surprising fact or relation. And I can ask the programme to specify as far as possible the conditions of the unexpected fact or relation, for example: is it incidental or structural? Thinking of a brainstorming support interface, I think this conclusion will help.

Making flat table interactive with Kinect and a projector

Microsoft Kinect is more than a game device, with the help of the embedded depth sensor, one can turn any flat table to an interactive multitouch surface. I made it yesterday. Just hang it for more than 1 meter above the table, and project your interactive content onto the table. That’s all for the hardware part.

For the software, you have to use OpenNI(or other Kinect hacks such as freenect), OpenCV to get your touch finger blobs,  this needs some programming.  You have to fire TUIO events when touches are detected to a multitouch-enabled application. That’s it. For the application, I use MT4j for a demo.