The power of the “what do you think?” question

“What do you think?” was once an internal joke where I used to work. During during lunch time, my colleagues would laugh whenever some one asked “hey how does _x thing_ work?”, because they would immediately answer “what do you think?!” Truthfully speaking, they were making fun of me and my silly questions during the usability tests, and after a while I was just proud that they learned so well.

My coworkers thought it was funny, but also a little bit evil or rude, that I answered the customers’ questions during a usability test or research interview with another question. This was specially awkward for them because they had very good answers for all the customers’ questions, like amazing features the test device could do, but their words were trapped in their mouths, when they were shushed by my question “What do you think?”.

As a matter of fact, I was very knowledgable of the medical device too, not only all the can do’s and dont’s, but the actual stuff that was not implemented yet and required the skill of simulation and deception during the usability tests. I just refused to provide immediate answer to the direct customer questions, because what is the use of that?

As a HFE / UX researcher, your attitude influences the interviewee or usability test participant…

There is a lot of value on user research to inform the design and development of a medical device. Have you stopped to think about your attitude during a usability test? I believe my colleagues learned about the “what do you think?” attitude and the value it provides.

The question served my research team in two different ways:

1. During Formative Evaluations, when we were testing the usability of a medical device prototype, we often answered back with this question to collect the first and non-biased impressions about the device design, workflows and value proposition. Most importantly, by hearing these impressions, we could determine if there were any mismatches between the user’s perception and intuition about the device functions and the actual intended design and use.
2. During Summative Evaluation, this question was a shield, a reminder that the researcher was there to observe and facilitate the evaluation and not to help. When the participant was reaching a dead end, and it was evident that an intervention was required to continue using the device, this question would support the assistance. It allows the participant to think out loud the issue experimented and potentially serve as way to determine on their own other potential solutions.

My colleagues laughed about this question, and found themselves using more often, and mostly for the two purposes above. They realize that the information and value of the usability tests are richer when the participant is allowed to speak first their thoughts.

As a HFE researcher, I realize that the question places me on another level in front of the interviewee. With this subtile and short interaction, any person participating in the usability evaluations, adapts to the idea that we are there to listen and not to demonstrate and explain how the device features.

I will follow up on the how the researcher attitudes may shape the usability test dynamics in the next articles.