User research process

An overview of how we work and what to expect

The eAPD team aims for research to be fully baked into our process so that it doesn’t feel like a separate “research process”, but just a thing that the team does as part of our normal routine. We have an open invitation to the team that anyone can attend and observe the research sessions. This helps create not only a strong understanding of the users of our product, but a bond on the team around user needs.

Research plans are made by design in collaboration with product, SME, and dev if needed. The plans are available for the whole team to provide feedback on either via Mural or Google Docs.

We aim to do a research study at least every 4-6 weeks at minimum.

Recruiting and scheduled has typically done by SME who is part of the APD CoP.

Sessions are open for observation by the whole team and are generally facilitated by design, with some exceptions for product and SMEs to facilitate. Everyone is also invited to a debrief after each session, but design, product, and dev lead are required (lite required if there is a conflict).

We are currently using Dovetail to do synthesis and analysis, which includes the ability to upload a video, get transcripts, highlight and tag comments, and add insights to tags and comments.

We generally connect to users via Zoom unless they have any restrictions on video tools. The facilitator and participants are encouraged to be on camera while note takers and observers are encouraged to remain off camera.

Typical goals of user research that are team specific (vs does it work, is it usable) seek to answer the questions: 

1. Will this content, feature or change support getting an approvable APD out of our system 
2. Do states understand what CMS wants from them to make the APD approvable

We’ve done exploratory user interviews, usability tests, group conversations with states around collaborative processes, a pilot test, beta testing for our upcoming release, which also included a diary study where folks would spend time in the application and take notes on their experience with a debrief after. A full history of user research studies is on this wiki page.

A document with research checklists and deeper process information can be found in the eAPD google docs repository.