When designing usability tests, our clients often think of them as Focus Groups, and want to use the opportunity to ask a whole series of exploratory questions.
I don't mind including some, but I save them for the end, and I edit them heavily. Jared Spool of User Interface Engineering has written a beautiful article, 3 Questions You Should Never Ask During User Research, that succinctly explains our approach. All kudos to Jared for his group's leadership in the usability/user experience space.
Users are Experts on their Pain
The reason we do user testing with designs and live sites is to see any user pain or struggling directly. It's hard to sit still and watch someone struggle, but it's essential to getting the information we need to make the site experience better.
Whether the project is a site redesign or a full process reengineering, we look at it this way. The user is the patient and the UX designer is the doctor. Patients are expert about his/her own pain, emotional state and relative happiness; no one else can know or measure that as accurately. Patients are NOT expert on diagnosis, or creating and implementing a treatment plan. The doctor does that.
Step 1: Self-diagnosis. "Doc, I think I have cancer!" says the patient.
Step 2: Explore for information. The doctor calmly responds, "Why do you think you have cancer?"
Step 3: Specific goals, needs and requirements. "My stomach hurts right here after I eat. And my mother's uncle died of stomach cancer!"
Step 4: Diagnosis and treatment. After delving into the specifics, the doctor recommends a course of action. "Take this acid preventer before each meal for 2 weeks - and let me know how you are feeling. You should feel relief by tomorrow at dinnertime. If you get to feeling worse at any point, or a specific, sharp, stabbing pain in this region, please call us back right away."
Step 5: Measure success, adjust treatment plan. After a course of treatment, either the patient's complaint is addressed, or the problem is still present. Either they're done, or both continue to work the problem.
User-Centered Design Example
So let's take that medical example and make it into a website marketing example.
Step 1: Self-diagnosis: The marketing director says, "We need to set up a CEO blog right away!"
Step 2: Explore for information. "Why do you want a company blog?" I respond.
Step 3: Specific goals, needs and requirements. "We need to be seen as cutting-edge. We need to be driving more traffic to the website. And XYZWidgets has a blog, and they're kicking our butts!"
Step 4: Diagnosis and treatment. Now we can explore options for solving the problem. I might say, "Let's identify some good, juicy content topics that you can make available to prospects in a variety of ways - website, email, social media. We can include an excerpt in the next eBlast campaign, post it on your Facebook page, and drive prospects directly to a landing page for the full article. Let's see what the response is to that before investing your CEO's time in a blog." We set specific goals at this step.
Step 5: Measure success, adjust treatment plan. This step is critical, and it's often overlooked or underperformed by busy marketers, and even more frequently by folks who are doing marketing part time, as one "hat" in addition to another primary role.
I highly recommend subscribing to Jared Spool's UIE Tips eNewsletter. They're usually good; today's was excellent!
