Qualitative vs. Quantitative Evaluation

Definition

When measuring User Experience or Usability you use qualitative and/or quantitative research [1, 2, 3].
Qualitative Data are insights or observations regarding your users' UX or usability (subjective feedback). You will be able to directly watch your participants and follow-up with problem-specific questions. It helps you to find problems and design solutions.
Quantitative data is gathered through measurements related to UX or usability (e.g. task completion rate or time). You can use it to compare several designs or identify the most pressing issues (statistically).
- “Quantitative metrics are simply numbers, and as such, they can be hard to interpret in the absence of a reference point. For example, if 60% of the participants in a study were able to complete a task, is that good or bad? It’s hard to say in the absolute. That is why many quant studies usually aim not so much to describe the usability of a site, but rather to compare it with a known standard or with the usability of a competitor or a previous design.” [3]
Mixed-MethodResearch : In UX research, it is common to use both qualitative and quantitative data. While quantitative data answers the question “how many and how much?”, qualitative data answers the question “why?” which enables you to analyse and explain your quantitative data more effectively. They work well together as a team.

The table by the Nielsen Norman Group [3] provides a very good summary, we allowed ourselves to alter the text a bit:

	Qual Research	Quant Research
Questions answered	Why?	How many and how much?
Goals	Both formative and summative: inform design decisions identify usability and/or UX issues and find solutions for them	Mostly summative: evaluate the usability and/or UX of an existing site track usability over time compare site with competitors compute ROI
When it is used	Anytime: during redesign, or when you have a final working product	When you have a working product (either at the beginning or end of a design cycle)
Outcome	Findings based on the researcher’s impressions, interpretations, and prior knowledge	Statistically meaningful results that are likely to be replicated in a different study
Methodology	Few participants (around 5) Flexible study conditions that can be adjusted according to the team’s needs Think-aloud protocol	Many participants (at least 40) Well-defined, strictly controlled study conditions Usually no think-aloud

[1] - Moran, Kate. “Quantitative Research: Study Guide.” Nielsen Norman Group 8 (2021).

[2] - Moran, Kate. “Qualitative vs. Quantitative UX Research.” Nielsen Norman Group 2 (2020).

[3] - Budiu, Raluca. "Quantitative vs. qualitative usability testing." Nielsen Norman Group 1 (2017).