A measurable usability goal is the definition of successful usability on your site for a specific set of users doing a specific task.
You are building a Web application for owners of small businesses to pay state withholding tax online. This will allow a business owner who has been paying withholding tax by mailing in a check and slip every month will now pay that tax online. On the first attempt, the business owner will:
- successfully complete the transaction in five minutes or less
- submit the right amount from the right bank and bank account
- make no more than one error while using the application and recover from any error in one minute or less
- rate the experience a four or five on a one to five scale where five is the best
- indicate a desire to use the application for future tax payments
Typical usability goals include time, accuracy, overall success, and satisfaction measures.
You can set a usability goal for the overall time the user will take to carry out a task (scenario) on the site. You can also break down that time and set measurable usability goals for:
- time to get to the application or to the right Web information page
- time to use the application or to understand the information
- time to recover from an error
Similarly, you can set a usability goal for the accuracy with which the user carries out the task (scenario) or you can break it down into separate goals for:
- number of unproductive navigation choices
- number of unproductive searches
- number of errors in using an application
- number of misunderstandings of information
Obviously, the usability goal must be that users will be successful. If users cannot do their tasks or cannot get answers to their questions on your Web site, your Web site is failing those specific users for those specific tasks and questions.
You may also set measurable usability goals for how users will get to that success. For example, you might set a measurable usability goal for a Web application that new users will go to the help if they need it, will find what they need in the help, and will be back doing their original task within two minutes. You might set a measurable usability goal that a user who has done the task in your Web application before will do it successfully a second time without using the help.
Again, obviously, your measurable usability goal must be that users are happy. You can measure overall satisfaction. You can also break down satisfaction and set separate measurable usability goals for navigation, search, level of detail in the content, language of the content, and other specific factors.
Relying on the Measures
When you test the Web site against your measurable usability goals, consider performance (time, accuracy, success) as more important than satisfaction ratings. If users give the site low ratings, the site needs to be fixed. If users give the site high ratings, you may not be getting a true picture. In usability testing, we often find that users give high satisfaction ratings even when they have had serious performance problems. They may be blaming themselves for the problems. They may not want to hurt your feelings. They may be being polite rather than saying what they really think.
A good way to set measurable usability goals is to observe users doing similar tasks or seeking similar information today.
If you are developing a Web application, your application must allow users to do their tasks at least as fast with as few errors and as much success and satisfaction as their current way of working. Ideally, it should let them be faster, more accurate, more successful, and more satisfied. Otherwise, they won’t use it.
If you are developing an informational Web site, your site must outperform other ways users get the same information—phone, printed material, and other Web sites.
You can also use Usability Testing to set measurable usability goals by doing a usability test of your current site. You should record data on potential measures including time, accuracy, success, and satisfaction. If users have problems during the test then set specific goals to improve each of the problem areas.
When you set measurable usability goals, you must focus on what is going to make users work effectively and efficiently and be satisfied. You cannot say, “the system response time is going to be very slow, so we will set our time goal to account for that slow response.” Users will leave your site if it is too slow. You must set a goal that matches users’ needs and expectations and find a design solution to improve system response time if that is going to keep you from meeting the usability goal.
Web site level
- 95% of customers will be able to find and order a product.
- 95% of physicians will be able to find, read, and understand the latest information on lung cancer treatments.
- 95% of travelers will be able to make their own airline reservations.
- All trained “service representatives” will be able to handle an average of 25 customer calls per hour.
Scenario level usually refers to two or more page and addresses issues related to one major type of user interaction). For example:
- 90% of users will be able to find a specific article on thyroid cancer within three minutes.
- 90% of users will be able to read an “update” article on skin cancer in less than five minutes.
- 90% of users will be able to make an airline reservation in less than five minutes.
Page level is always within a page and is usually the homepage. For example:
- 90% of users will be able to find and click on a specified link within 15 seconds.
- 90% of users will be able to find and click on a specified graphic within two seconds.
- The page will download in five seconds or less on systems using a broadband connection.
- 95% of users will be able to use the dropdown selection box to find and select a specific color.
- 95% of users will be able to use the alphabetical list to select information on lung cancer.
- 95% of users will be able to use radio buttons to make a selection.