Usability Test Heat Map with Difficulty Level added

I dug a little further into the data from my recent paper prototype usability test for GNOME Settings.  I wanted to add some additional information on the experience of each participant of the Usability test beyond whether or not their GNOME settings choice was correct..  Below is the original heat map that I did, showing whether or not the participants correctly linked the scenario tasks with the appropriate settings.  Green represents correct choices, red incorrect, and orange represents two part choices where one was correct and the other incorrect.  I added in some participant demographics so that the results could be more easily compared to the profile of test participants.  I also added difficulty level to the analysis within each cell of the heat map.  Difficulty is represented numerically with numbers ranging from 1-3, 1 being easy, 2 being moderate, 3 difficult to complete.  Difficulty level was determined by time needed to complete the task and the level of confidence/wariness conveyed by the tester in making their selection.  Hopefully this will give you a better sense of the process behind the task attempts.

 

Key Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7 Test 8 Test 9 Test 10
Age 37 30 27 28 21 20 31 39 28 29
Gender man Non-binary man woman woman man woman man Non-binary man
Occupation web dev app dev student musician student game dev Business sys admin Call Center UPS printing
OS Linux Linux Mac Windows Windows Windows Mac Linux Windows Mac
Computer usage at work heavy heavy light light light heavy moderate heavy heavy light
Computer usage at home heavy heavy moderate light light moderate light heavy light moderate
Previously used GNOME Yes Yes No No No No No Yes No No
Previously used Ethernet? Yes Yes Yes No No Yes Yes Yes Yes Yes
Alarm Volume Sound 1 1 1 1 1 1 1 2 1 1
Monitor Mirror Devices, Displays 1 1 2 1 2 1 1 2 2 2
Default Email Client Details, Default Applications 2 1 2 1 2 1 1 2 1 2
GNOME Version Details, About 1 2 2 2 2 1 2 1 1 1
Bluetooth Connection Bluetooth 3 2 1 2 1 1 1 1 2 2
Monitor Colors Details, Color 2 1 3 2 2 2 2 2 2 2
Search Index Search 1 1 2 2 1 1 2 1 1 2
Keyboard Speed Devices, Keyboard 1 1 3 3 1 1 1 2 1 1
Switch Language Region & Language 1 1 1 1 1 1 1 1 1 1
Set Default Video Player Details, Default Applications 1 1 1 2 3 1 1 2 3 2
Battery Life Power 1 1 1 1 1 1 1 1 1 1
Projector Connection Devices, Displays 1 1 2 1 2 1 1 2 2 1
Screen Sharing Sharing 2 2 2 1 1 1 1 3 2 3
Screen Lock Time Privacy 3 1 2 3 3 2 2 2 2 1
Printer Connection Devices, Printers 1 1 1 1 1 1 1 1 2 1
Time Zone Date & Time 2 1 1 1 1 1 1 1 1 1
Mute Notifications Sound 1 2 2 1 1 1 2 1 1 2
Alert Messages Notifications 1 2 1 1 1 1 1 1 1 1
Lock Screen Image Background 1 1 2 1 1 1 1 2 2 1
2 Finger Scrolling Devices, Mouse & Touch 1 1 2 3 3 1 1 2 2 1
Find Saved WIFI Pass WIFI 1 1 1 1 2 1 3 1 2 1
Ethernet Connection Networks 1 1 1 3 2 2 1 1 2 1
Switch WIFI Networks WIFI 1 1 1 2 2 1 1 1 1 1