The Five Phases of User Testing
Guest Post on Mindtribe Blog | August 2018
After years of practicing user-centered product development, I redesigned the way Moxxly approaches user testing with prototypes. In a recent post, I outlined the seven principles that came out of this redesign, and highlighted how to automate your workflow to get the most of our user testing.
Today, I want to dive deeper into one of the seven principles: Define the Test’s Phases.
By implementing this one practice into your testing cycle, you’ll get the most from every test and close the loop with your findings, keeping your team up to speed and your work integral to the product cycle.
If you test as much as we do, you’re always testing! Before our user testing process redesign, some tests would begin to bleed into each other and the results across variables began to blur. No more!
Now we make sure that each round of testing has clearly defined phases with a discrete beginning and end. The beginning of every round of testing starts with the question you’ve built the prototype to answer and ends with a presentation to the team with the answers to that question.
Here are Moxxly’s Five Phases of Prototype User Testing:
1. DESIGN THE TEST: CREATE A TESTING PLAN
What’s the one question you’re trying to answer? What are your assumptions? What are you testing? Why? How? What does success look like and what are the next steps if it is / is not successful?
For example, let’s say your team is developing a new sensor technology. The benchtop results of the tech are promising, but the team hasn’t really thought about the user experience yet. After a few brainstorms and prototyping sessions, you have six different interaction prototypes and lots of assumptions about how they will be used. Now’s the perfect time to get user feedback to help inform the product direction.
Your user testing plan might look something like this:
Why are we doing this?
To get a better understanding of our users attitudes and behaviors around different interaction prototypes and to learn what is intuitive and what is confusing about the interaction prototypes.
To get a better understanding of our users attitudes and behaviors around different interaction prototypes and to learn what is intuitive and what is confusing about the interaction prototypes.
What are our goals?
To conduct 45 min feedback sessions with 5 – 10 San Francisco-based women on their behaviors during their pumping journey to help inform our sensor interaction design experience.
To conduct 45 min feedback sessions with 5 – 10 San Francisco-based women on their behaviors during their pumping journey to help inform our sensor interaction design experience.
With whom do we need to test?
5 – 10 women
San Francisco-based or can meet in San Francisco
Currently pumping or have stopped pumping in the last 3 months
5 – 10 women
San Francisco-based or can meet in San Francisco
Currently pumping or have stopped pumping in the last 3 months
What are our assumptions?
Women do not want to touch milk-covered parts
Women do not want increased height on their system
Women do not want to deal with the “hassle” of cleaning
Women do not want to touch milk-covered parts
Women do not want increased height on their system
Women do not want to deal with the “hassle” of cleaning
What is our timeline?
3 weeks:
3 weeks:
1 week of planning and recruiting
1 week of testing
1 week of analysis
1 week of testing
1 week of analysis
How will we know if we’re successful?
If we complete all the interviews and see patterns emerge that help lead to a POV on the user experience of the sensor
If we complete all the interviews and see patterns emerge that help lead to a POV on the user experience of the sensor
Next steps if test is successful
Share learnings with team to help inform Works Like-Looks Like prototype direction
Share learnings with team to help inform Works Like-Looks Like prototype direction
Next steps if test is not successful
Test with more women.
Test is not successful if patterns do not emerge or if we don’t test with at least 5 women.
Test with more women.
Test is not successful if patterns do not emerge or if we don’t test with at least 5 women.
2. RECRUIT: DEFINE YOUR TARGET AUDIENCE
Who’s your target audience? Make a list of necessary user attributes and, depending on your budget, either work with an agency to find her or get out to where she and her posse congregate (digitally and physically). Most of our users have signed up for testing on our website, so if you don’t already ask for sign-ups on your site, set it up!
3. TEST!: FOLLOW TIGHT, REPEATABLE STEPS
All your prototyping and prep work culminates in this moment. So get the most out of it by having – and following – tight, repeatable protocols to ensure validity across tests, including reducing variables as much as you can when dealing with real people and introducing randomness to avoid order-based preferences.
Walk into the test with a field guide that makes it easy for you (or ideally a partner) to take notes, capture insights, and compare results across users. Without leading questions or prompts, ask her to interact with the prototypes and have her narrate her thoughts and interactions as though narrating a film. When she says, “Oh … interesting” or “That’s cool!”, dig in and ask her what she means. If her actions contradict what she’s saying, don’t be afraid to get clarification. After she’s interacted with all the prototypes, have her rank them based on different judging criteria, e.g. overall preference, ease of use, ease of assembly, aesthetics, or cleanability. Most importantly, ask her why she ranked them the way she did. Make sure you understand where she’s coming from, what she cares most about, and what she’s willing to give up.
4. ANALYZE: KEEP ASKING THE “WHY”
After you’ve conducted the feedback sessions, dig into the data. There are many ways to analyze and make sense of user feedback sessions.
Here are three tools relevant to our interaction prototypes testing example:
User Journey: Mapping out a user journey is a great tool to illustrate how and in what order people interacted with the prototype and to point out the failure modes and design opportunities for each design.
Affinity Diagramming: Cluster interesting user quotes and behaviors and title the clusters. This will lead you to key insights and design principles that will influence the next prototypes.
2×2 Framework: Categorize the prototypes and map your users’ top two overall preferences.
Be curious, dig in, and ask why, why, why until you’ve made sense of the mess.
5. SHARE THE FINDINGS: ALWAYS ASK FOR FEEDBACK
Presenting your findings to the team not only keeps them up to date on user testing, it also creates a precise ending to each round of user testing and a chance for you to reflect. Did you do what you say you were going to do? What went well? What process improvement can you implement next time?
End by asking for feedback, always.
YOUR TURN
Moxxly’s Seven Principles of User Testing is a result of adapting the d.school process, swapping notes with other startup designers, and years of practice and iteration with real women. It’s a living document, undergoing continuous process improvements. If you’ve brought this process back to your team and have feedback for me, I’d love to hear it.
__
This article was originally published on the Mindtribe Blog.