Manual

The functions of Novice–Expert+

By Karl F. MacDorman on June 1, 2022

Purpose

This extension identifies usability issues with a website or web app. The extension logs the steps a user takes to perform a task and estimates the time an expert would take to complete each step. This information may be stored to a CSV file, so that novice and expert performance may then be compared. The novice time divided by the expert time is the novice–expert ratio. If this ratio is above the average, it could indicate a usability issue.

The extension performs the following functions:

  1. Determine Fitts' law constants for expert pointing time using a particular device (e.g., Mac trackpad, Logitech optical mouse).
  2. Log a user's web activity in terms of elementary operators like click, keystroke, and scroll.
  3. Estimate expert performance for the logged activity.

This tutorial assumes knowledge of KLM GOMS.

Installation

  1. Click Novice–Expert+ to install the extension from the Chrome Web Store.
  2. Click Add to... and then Add extension.
  3. Click on the browser's extension icon and pin the extension so that its icon remains visible in the extension bar.

Pop-up

Click on the Novice–Expert+ icon to open its pop-up. The pop-up will only open if the browser tab is open to a website or webpage.

The pop-up has an Actions panel, the user's total time to complete the task, and a row of four buttons: Record, Clear log, Save as CSV, and Options. 

Actions

The Actions panel displays user-initiated events. These are the steps the user takes to perform a task. The extension records when each event occurred to determine how long each step took to complete. It also estimates expert execution time for that step.

Expert actions are home hands, mentally prepare, and wait for system response. Home hands is automatically inserted when the hands first use the keyboard or pointing device or when the hands move between the devices. Mentally prepare and system response are manually inserted by clicking between actions.

Total time is the total time it took the user to complete the task.

Options

Clicking on Options from the pop-up opens up the options page. The following time constants may be updated: Fitts' law a and b, mentally prepare, home hands, and system response. All values are in milliseconds.

Clicking on Determine a and b from the options page opens a page containing the Fitts' law experiment and regression graph.

Fitts' law

Experiment

The experiment presents the user with a block of trials. In each trial, a dot immediately appears, and the user must click on it. As soon as the user clicks on the dot, the next trial begins. The dot is green in the first trial in a block and red in the remaining trials.

The width of the dot is varied systematically across trials as is the distance between dots in successive trials. Thus, width W and distance D are independent variables in the experiment, and together they determine the index of difficulty ID:

LaTeX: ID = \log ( \frac{D}{W} + 1)

The dependent variable is the time t the user took to click the dot. The experiment results in a list of ID, t pairs: (ID1, t1), (ID2, t2), ... IDn, tn). When the user pauses, these points are graphed with a regression line.

Regression graph

The regression graph displays the data from the experiment, plotting the IDs on the x-axis and the ts on the y-axis. A line is fit to the data by least-squares regression. The y-intercept is the constant a, and the slope is the constant b:

LaTeX: MT = a + b~ID

These constants enable expert pointing time to be estimated by Fitts' law:

LaTeX: MT = a + b \log(\frac{D}{W} + 1)

The Pearson's correlation coefficient r indicates the strength and direction of the relation between ID and MT. Although values between –1 and 1 are possible, in the context of the experiment, a strong positive relation is expected with a value for r of 0.8 or higher.

If the actual time recorded for a trial differs from its estimated time based on the regression line by at least two standard deviations, it is marked as a potential outlier. The trial may be removed by clicking on the point. It should only be removed for a valid reason—for example, if the user was interrupted during the trial.

By pressing Update a and b, the values for a and b derived from the newly collected data replace those currently in use.

The values for a and b depend on the particular system being used. For example, they will differ for a mouse, trackpad, or trackball. They will also vary with the input device's tracking speed and other settings. Therefore, it is important to update a and b before logging user-initiated events.

Record

After pressing Record on the pop-up panel, the extension begins to log user-initiated events. The user may then interact with websites or web apps and view the resulting events in the Actions panel. After completing any tasks to be evaluated, the button should be press again to pause recording.

Clear log

After pausing recording, Clear log may be pressed to delete all recorded events and expert operators. This information will be irrecoverable lost unless it has been saved.

Save as CSV

Press this button to save all logged actions as a comma separated value (CSV) file. The file may be opened in a spreadsheet or other application for further analysis.