User testing. Why bother?

OK, so you want to build a better product. You might have heard anecdotal complaints about it around the office. Perhaps you have an idea for a new ground-breaking feature. Maybe it's just not up-to-scratch from a personal point of view.

Whatever.

Without quality research or testing, you are essentially basing your designs on assumptions. If you don’t take the time to engage with real users, it’s virtually impossible to know what the real needs and pain-points your product should address. What I should point out right now is that testing should take place with participants outside of your immediate team and ideally with real target users. Otherwise, results will be biased and skewed with the vested interests of those people heavily involved in the project coming to the forefront. Similarly a fresh pair of eyes never hurt anyone as teammates too familiar with a project may miss some of the smaller, or larger, details.

User (UX) testing is the process by which you aim to understand the thoughts, feelings and actions of the people who use your application or service, obtained by either observing what they do or by asking them specific questions tailored to gathering the feedback you need. This can then all help you to either understand your customers better for future product enhancements, or for uncovering potential issues in new or upcoming software releases. Most importantly of all, it can help you find fixes and create better, more usable products. Note, how someone uses your product might be widely different to how they "think" they use it, hence the need to test and quantify these things.

Common perceptions are that user testing is costly and time consuming and when project teams are desperate to get software released, it's a tempting step to skip. However, that is simply not the case. Testing can be as simple or as complex as you want to make it (see guerrilla testing below) and the cost benefits of catching, and fixing, issues early before they are released to a wider audience can actually prove more cost-effective.

“If you think good design is expensive, you should look at the cost of bad design.” — Dr. Ralf Speth, Chief Executive Officer, Jaguar Land Rover

OK, so you're sold. No what? Well the first stage of any user testing activity should be planning - ask yourself what do you want to learn; what goal do you want to achieve; what are you trying to improve? From that you can determine your question, or hypothesis, and develop test methodologies to achieve the right results.

User testing methods, as a whole, can be split into essentially two types: qualitative and quantitative testing.

Quantitative testing is the achievement of metrics through data, generally speaking, at quantity. The cat food manufacturer who frequently expressed '8 out of 10 cats preferred it' famously knew this all too well. If 80% of your target audience are favouring your brand, then you're clearly onto a winner. Typically, the greater the sample, the more accurate your data and the more persuasive your argument can be.

Qualitative testing refers to any kind of research where the results take the form of observations, comments, thoughts and feelings and allow you to tell a story about the participant’s experience. If quantitative answers the “what” then qualitative is the answer to “why.”

Put simply, qualitative research can answer why or how to fix an issue, however quantitative research is better at answering how many and how much.

Types of user research & testing methodologies

Here are examples of some of the most common techniques used to carry out UX research. Many of the following methods can be run as either moderated or unmoderated and can be used to gather quantitative or qualitative feedback depending on how you set up your study. Ultimately whichever technique you employ is determined by your initial hypothesis.

Surveys and polls

A sample user base is asked, simply, to complete a questionnaire or survey and the results are collected and analysed. Typically used to get conceptual or general feedback on a subject, these can be quick to set-up and straightforward for the audience to complete.

Guerilla testing

This is probably your quickest and dirtiest type of user testing. Typically it involves setting yourself up in some public place, such as a coffee shop, and appropriating users as they pass by, perhaps with the incentive of a free drink. Once you have their attention, the moderator is then looking for quick and instant feedback on their product as users will probably only be able to spare 5 or 10 minutes. Guerilla testing works best in the early stages of the product development process. When you have a tangible design (wireframes or lo-fi prototypes) and what to know whether you’re moving in the right direction or not.

Click testing

A quick and simple way to test, on static images, where a user would click. This can be used in order to validate visual site design and/or user preferences using anything from sketches to high-res UI designs.

A/B testing

Also known as split testing or bucket testing, A/B testing is a method of comparing two versions of a webpage or app against each other to determine which one performs better. Testing takes the guesswork out of website optimisation and enables data-informed decisions that shift business conversations from "we think" to "we know." By measuring the impact that changes have on your metrics, you can ensure that every single change produces positive results.

Case study: The £1M button: An eCommerce group had the hypothesis that subtly changing the text on their 'Add to basket' button on product pages would increase conversions. A simple A/B test was ran for a few weeks with 50% of the site visitors seeing the old button and 50% being served the proposed button. After the test had ran, clickthroughs for both versions were measured and subsequent conversion was so great that if the new button were rolled out fully, it was predicted to make the business over £1 million in additional revenue.

Usability testing

This method requires having users interact with your prototype, website, app or any other digital product in order to capture their feedback, actions, behaviours and/or spoken-aloud thoughts. Lab usability testing works best when you need to have in-depth information on how real users interact with your product and what issues they face. The fact that this testing is moderated enables you to collect more qualitative information. However, with lab testing, there is always a risk that the controlled environment will be different from the user’s real environment and by placing the user in a controlled atmosphere, there is always a risk of creating a non-realistic user behaviour.

Remote usability testing

Test participants are asked to complete tasks in their own environment using their own devices and without a moderator present, which leads to the product being used naturally. The cost of unmoderated testing is lower than lab testing; however, this type of testing offers less detailed testing results. Unmoderated remote usability testing works the best when you need to obtain a large sample to prove critical findings from your initial moderated research.

So, there you have it. A quick guide to why you should test and how to go about it. However don't forget that user needs and preferences change over time so the more you are able to embed a research and testing schedule into your product development cycle then the greater your chance of shipping quality, user-friendly experiences first time, every time.