Digital Strategy

Getting Started with User Testing for Website Optimization

Why do we need user testing?

The answer is really simple. You want to create a successful product. Your website will perform ten times better if you develop an efficient and convenient experience for your users, not for yourself. Your users have to understand your website and feel comfortable using it, not you. User testing is the easiest way to find out if you meet the needs and problems of your users. Is the process or solution understood by your users, or are there still problems that you can fix. This will save you a lot of resources and time in the design and development process. You will solve problems that real users have, not the ones that you think you should solve.

How is user testing done?

Many people think user testing is very time consuming and connected to bigger investments, like setting up a proper usability lab and paying for users to come to the tests.
But testing can be much simpler, e.g. guerilla testing with a mobile phone on the street or just using a computer with a web cam and testing in your office or in the users natural environment. Often you will manage to find testers for free or very little money.

To integrate your users in your website design and development process most efficiently, tests should ideally happen every week, or at least every two weeks, integrated in your development process or sprint.

The process of user testing

Create hypothesis

What should be tested? It is crucial to understand that not everything can be tested. Try to identify a few main questions or tasks for the test, ideally together with your development team. One way could be to use the analytics data of your website or digital product to find problems. The data might for example show that there is a high number of people who do not finish the checkout. So in this example one potential hypothesis statement that you want to test could be: “We will achieve an increase in conversions by XY%, if our e-­‐commerce users successfully achieve their product purchases with the checkout design.”

Identify and recruit your testers

The test users have to be part of your target group. They need to be as close as possible to the persona that should have been created in an earlier stage. Another crucial point that we have to understand: we are looking for critical users, who do not give us answers that we want to hear. We are looking for users who will help us detect problems with the solution to make it bullet proof. Try to calculate around 30 Minutes to max. 1 hour per user and allow for enough time in between.

A few users are enough to see what works. More tests are not automatically better. A study by Jakob Nielsen showed that with only 5 test persons you can identify 85% of the problems. To learn more about this, read the article by Nielsen Norman Group: How Many Test Users in a Usability Study?

Come up with a test script

In order to ask the same content to all participants and to have comparable results, try to script the whole test scenario. Think of it as your interview guide. This will help you to cover all topics within the test. Every test will be different, but you should have always asked for the same feedback from every tester.

Split your team into research pairs

The teams of two should be mixed up, coming from different disciplines. One person takes the role of the interviewer, while the other person takes notes. These roles could switch at the end of the interview, to give the second person the chance to ask follow up questions.

Your prototype, the product to test

I won’t go into too much detail here. But this is a crucial part of testing. Our Minimum Viable Product (MVP) will help us to test our hypothesis. By minimizing the effort we put into unproven ideas, the sooner we are able to determine which features we should invest our limited resources in. The saying “Fail as early and often as possible!” describes the mindset of Lean UX and user testing quite well.

Running the user test

There is a great quote from an InVision blog article “At a user testing session, you’re the host. It’s your job to make people comfortable.”
Most testers will never have done this before and don’t know what to do or expect. So start by explaining what you are trying to do. It is really important to make the people understand, that they are testing our design or product and not that we are trying to test them. Make them feel relaxed by telling them that there is no wrong answer.

Let the tester talk

During the interview the method “thinking aloud” is used, so the tester says what he or she feels while trying to complete the task. This gives you valuable insights during the test. If the person might not get into it right away, you can keep asking “what are you thinking” to make them talk about their thoughts.

Ask open questions

Keep in mind to ask open questions like in any qualitative interview. Ask “why” often and try to understand where their behavior and answers come from. This will bring you real answers, that are not lead by your question.

Do not explain the prototype

The prototype or MVP should be introduced later on in the interview. Give the tester enough time to interact with it. But never explain the prototype. The prototype has to be self-­‐explanatory, this is a basic rule of user experience.

Never defend the prototype

When you get critical feedback about your product, do not try to defend it. Through testing in early stages, we don’t run the risk to fall in love with the prototype. This makes it a lot easier to accept failure of the solution or of parts of the solution. Failure has to become a normal part of your development process.

Answer questions with counter questions

By using counter questions, the tester has to come up with a solution on his or her own.

Make the most out of the results

Visualize the results and present the results to the design and development team and other important stakeholders.
Brainstorm on potential solutions and improvements. Decide on next steps. Make the most out of the teams brainstorming ideas, by using them during the next iteration. Or by adding them to your backlog.
Iterate on prototype, test again or implement the new feature during the next sprint. Test again and again!

About the author



Currently in the Digital Business Strategy Master program at GEM, graduated from LMU Munich in Business Administration.

Digital Marketing Manager @ Dynafit, a mountain sports company.