phaseIII
Users want to have highly customizable, lightweight, and distraction-free new tab option for their browsers. We focus on developing and conducting user research to analyze our functional prototype.
Third Sprint Goal: Test the prototype for usability and satisfaction.
We conducted a summative study in order to measure the usability of the current prototype. We did not adopt a think-aloud protocol, however we did encourage participants to voice confusion or complaints. Tasks were presented in-order, however failure to complete one task does not impede another.
The prototype used during the study was a functional production build from our Software Engineering team. The current deployment can be viewed here.
We had (n=6) participants engage with our prototype. Participants were asked a few background questions, listed below and in our protocol document. Then, the participants were asked to complete five tasks under our observation. We recorded whether or not the task was completed or not and asked the participant to rate how satisfied they were with the task interaction. Following the testing, the participants were asked debrief questions that are focused on the participant's integration with this tool in the future. The test was conducted on a pre-configured laptop that was reset after each test. Participants had access to a laptop touchpad or wireless mouse on a mouse pad.
- Have you used browser extensions in the past? Which ones?
- What web browser do you use most often?
- We plan for this app to be available as a browser extension. Would you know where to look if you heard about it?
- You are looking at your homepage and dislike how it is organized. Try rearranging some
of the elements on the page and save your changes.
- Goal: Test discoverability of Edit mode (by pressing the Edit button) and basic widget manipulation.
- Add a new item to the homepage, position it to your liking, and save your changes.
- Goal: Test the Settings menu, discoverability of the Add menu, and the interaction of adding a new widget.
- You want to create a new page for work without losing your current setup. Try creating a copy of the current layout so you can come back later.
- Goal: Test the Settings menu, effectiveness of the 'Templates' verbiage, and the interaction of saving a new template.
- Remove some of the items from your homepage, and save your changes.
- Goal: Test discoverability of Delete mode and the clicking interaction.
- Try switching back to the layout you had originally.
- Goal: Test the ease and efficiency of switching between saved templates.
- Task completed: ⃞ Yes, ⃞ No
- On a scale from 1 to 5, where 1 is "Very Dissatisfied" and 5 is "Very Satisfied", how would you rate completing this task on this prototype? Why?
- Have you used any applications similar to this one before?
- What purposes would you mainly use this for?
- What features do you want to see added to this product?
All tasks were successfully completed by all participants, however some were not fully confident that they had completed task #3.
To display the data accuracy we decided to use Relative Dispersion Model (RDM) with slight modification to calculate the sets AADP (Average Absolute Discrepancy Percentage). Using AADP will provide us information how much data is scattered from the mean value. This will be applied for each task of the dataset we collected.
Average Absolute Discrepancy Percentage
- < 10% (Minimal Scatter) Highly concentrated
- < 20% (Low Scatter) well-clustered
- < 50% (Scattered) moderate dispersion
- >= 50% (Very Scattered) highly dispersed
| Task | Average | AADP | Feedback |
|---|---|---|---|
| t1 | 4.7 | 12.9 % | |
| t2 | 3.7 | 20.0 % | Many participants expected to be able to drag items rather than click them. |
| t3 | 3.5 | 20.0 % | Some participants found it unclear whether the app had successfully saved. |
| t4 | 4.5 | 11.1 % | |
| t5 | 4.7 | 10.0 % |
The study results show that the designed implementation is learnable with some places to improve, such as the design of storing layout settings. Participants enjoyed the visual aesthetics of the application and the interaction of manipulating widgets, however some found the Settings menu difficult to navigate.
Our recommended changes include adding a shortcut to the Add menu directly in the header bar in Edit mode, allowing widgets to be dragged directly from the Add menu, restructuring the Templates menu to be more explicit, and increasing visual feedback when the layout is saved or loaded.
Overall, we are happy with the positive user response and some even expressed interest in using the application outside of the study.
This study has the following caveats to consider and improve upon for future studies:
- Small number of participants (n=6)
- Limited participation demographic: all participants were Usability Engineering students.
- Some participants used a laptop touchpad, while later participants were given a wireless mouse.