Tutorial

Walkthrough

Consider the task of identifying a vacation rental in Indianapolis on Craigslist.

Record the user's actions

1. Start from https://indianapolis.craigslist.org/.
2. Press Record in the extension's pop-up.
3. Click on vacation rentals.
4. Get focus on miles from location.
5. Type 15.
6. Get focus on zipcode.
7. Type 46202.
8. Select Bedrooms: 1.
9. Click on 2br home in Bancroft.
10. Read the listing.
11. Click on reply.
12. Click on show email.
13. Click on the email address.
14. Press Pause in the pop-up.

System response and mentally prepare

Add steps for system response and mentally prepare before steps 3, 8, and 9.

Add mentally prepare before step 11.

Save and open in a spreadsheet

Save the log by clicking on Save as CSV in the pop-up.

Open the log in a spreadsheet.

Calculating the NE ratio

A table similar to the one below will appear in the spreadsheet. It will also include some extra columns, which may be hidden or deleted.

The novice–expert ratio is calculated for each step in a task. A step may include more than one KLM GOMS operator. There are several reasons for this. 1. Novice–Expert+ can only record user-initiated events. For example, if the user homes hands on the trackpad, mentally prepares, points to a widget, and clicks it, only the click is registered. So, the extension doesn't know when the transitions occurred between homing, thinking, pointing, and clicking and thus cannot calculate the execution time of each of these operators. 2. It is not useful to decompose a sequence of operators resulting from a single chunk of information. A word, for example, forms a cognitive unit, which results in a sequence of keystrokes when typed. Thus, the sequence should be treated as a unit. 3. Excessive granularity can make it difficult to explain the results to others.

The first novice–expert ratio is the time it took the novice to click vacation rentals divided by the expert time. We can't separate mentally prepare, point, and click time for the novice, but we know the time taken was 6231. The estimated expert time for home hands, mentally prepare, and point and click is 400 + 1200 + 1223, so the NE ratio is

The next click is to get focus on miles from location. The novice is not responsible for time spent waiting for a system response, so this time is deducted from the time taken. Thus, the NE ratio is

The next table entry is home. which indicates homing the hands on a device. The extension has detected a transition from using the pointing device to typing on the keyboard. A row is added for the 400 ms expert estimate.

We don't know how long the user took to home the hands, but we know the time taken to home the hands and type 15 is 1349 + 862 ms. The expert estimate is 400 for homing the hands plus 2 times 200 for hitting two keys. Thus, the NE ratio is

eventType	nodeText	width	height	centerX	centerY	timeTaken	expertTime	NEratio
home							400
think							1200
click	vacation rentals	224.8	22.55	784.79	336.53	6231	1223	2.21
response							70
think							1200
click		60	27	43	289	12334 –70	1253	5.00
home							400
keystroke	1					1349	200
keystroke	5					862	200	2.76
home							400	.
click		78	27	117.01	289	1120	1022	0.79
home							400
keystroke	4					1417	200
keystroke	6					418	200
keystroke	2					862	200
keystroke	0					273	200
keystroke	2					279	200	2.32
think							1200
home							400
click	0 1 2 3 4 5 6 7 8	51.5	24.5	38.75	538.75	4503	1180	1.62
response							70
think							1200
read							9600
click	2br home in bancroft	168.28	22.5	392.05	435.95	12037 –70	1211	1.00
response							70
think							1200
scroll		600	450	407.5	248.85	5308	1100
read							9600
scroll		600	450	407.5	467.85	16049 –70	1100	1.61
think							1200
click	reply	77.35	40	146.18	126.66	5138	912	2.43
click	show email	90.01	26.66	159.5	155.66	3301	864	3.82
click	@hous.craigslist.org	367.34	16	300.17	176.33	2966	1070	2.77

In this manner, we complete the remaining NE ratios.

Two rows are missing for the estimated expert reading time for 40 words at 250 words per minute:

ms

The NE ratio for scrolling and reading is

The average NE ratio is

Conclusion

The website presented several usability issues. After visiting Craigslist for Indianapolis and selecting vacation rentals, rentals appeared for Florida and other places. The 5.0 NE ratio before narrowing the search reflects disorientation with the unexpected results.

Another surprise was the need to click three times to reply to an ad with the third click opening an external email program. Show email had an NE Ratio of 3.82.