Quick Tips
We have collected an incomplete amount of short evaluation tips that might help you when planning and carrying out evaluations.
Provide Clear Instructions
Make sure that your evaluation tasks are straightforward and easy to understand. If participants need help understanding what they have to do they might feel lost, and you might not get the answers you are looking for. Furthermore, you might have to intervene during testing, which you want to avoid altogether, if possible.
Unbiased Questions/Tasks
Wording questions or tasks can influence the test results considerably. Take the appropriate time to formulate your questions or tasks in an unbiased manner and, if possible, check them with colleagues in advance.
Ask Open-Ended Questions
Respondents should be able to choose their own terms when answering questions. Make sure to avoid wording your questions in a way that participants can respond by saying “yes” or “no”. This will not help you to understand their thoughts and feelings better.
Be Careful Asking “Why” Questions
This kind of query implies a cause-and-effect relationship that might not actually exist. These questions may also make respondents feel defensive as if they must justify their answers, which could prevent them from providing honest replies to this and other inquiries.
Ask One Question at a Time
Refrain from overwhelming your participants with too many questions. Ask one question at a time. This will help your participants to keep focus, and you will get more detailed answers.
Pilot Testing
Test your evaluation in advance! Do not start with the real evaluation right away. Best case scenario: everything works, and you already have tested your first participant. Worst case scenario: certain things did not work, were not understood by testers, or you realised that you needed to ask more/different questions. This way, you will only “lose” one or two possible testers but not spoil the real user test.
Wrong Atmosphere
Testers have to feel comfortable. Do not put them on the spot or even criticise them! It is always helpful to emphasise that you are testing the system, not the participant. If a mistake happens, it is not their fault, but the system needs to be designed better.
Fail to Observe
A lot of faults of your system become apparent only during observation. If you have the chance, always observe your participants during testing. They might not think about certain things to say or even not realise them, but you will see them during the observation.
Overburden Testers
Preselect a small number of topics you would like to test, do not overburden the tester. It is often better to challenge three questions in detail than to focus on a large number of questions.
Take Notes
You will not be able to remember everything. Remember to take notes during your evaluation. Or, if you are recording the user test, write down timestamps of interesting situations.
Do not Tell the User What to Do
Do not tell users what to do even if they ask for help. Instead, ask them what they think they could do or where they would look for a solution if they get stuck.