What if we studied contradictions instead of patterns?

Most research synthesis works the same way. You run the interviews, write every observation on a sticky note, and cluster the notes until themes appear. Seven of ten people mentioned cost. Five of ten struggled with the setup screen. The themes become insights, and the insights become the roadmap.

It’s a good method. It’s also built to throw away exactly the observation that might matter most: the one that doesn’t fit any cluster.

The note that doesn’t fit

Every synthesis wall has one. A participant who loved the feature everyone else hated. Someone who used the product in a way nobody designed for. A person who said the opposite of what their behaviour showed. These notes tend to end up in a pile marked “other”, and then quietly stay there.

There’s a good reason for that. One person is not a trend, and designing for one odd case can mislead a whole team. But “not a trend” isn’t the same as “not important”. A contradiction tells you that your explanation is incomplete. If your theory says everyone finds the setup confusing, the one person who breezed through it knows something your theory doesn’t.

Why the wall buries it

It’s worth being precise about why the odd note gets lost, because it isn’t laziness. It’s the method working as designed.

Clustering is a form of compression. You start with three hundred observations and end with six themes, and the whole point is to lose detail. Compression keeps what repeats and drops what doesn’t. A note that shares nothing with its neighbours has nowhere to go, so it falls out first. Nobody decided it didn’t matter. The process decided for them.

Then there’s counting. “Seven of ten” feels like evidence, so we start treating how often something comes up as a measure of how much it matters. Virginia Braun and Victoria Clarke, whose paper on thematic analysis is one of the most cited in qualitative research, are clear that this is a mistake. A theme earns its place by capturing something important in relation to the research question, not by appearing in the most transcripts. They also push back on the idea that themes simply “emerge” from data. Researchers build them. That matters here, because if we build the themes, we also decide what gets left out of them.

The last reason is the hardest to see from the inside. By the time you’re at the wall, you usually have a hunch. Raymond Nickerson’s review of confirmation bias describes how reliably people seek, notice and weigh evidence in ways that favour what they already believe, and how rarely they look for evidence that would prove them wrong. A synthesis session is a room full of people with hunches, sorting evidence by how well it fits. The note that doesn’t fit is the one we’re least equipped to notice.

Other fields already do this

Qualitative researchers have a name for the fix: negative case analysis. You look actively for cases that disagree with your emerging explanation, and you revise the explanation until it accounts for them. The point isn’t to defend the pattern. It’s to test it.

The planning scholar Bent Flyvbjerg makes the logic sharp. If your claim is “all swans are white”, a thousand white swans don’t prove it, but one black swan is enough to break it. In-depth work is unusually good at finding black swans, he argues, because what looks white from a distance often turns out to be black up close. He also makes a point that should be pinned above every research team’s desk: the typical or average case is often not the richest source of information. Atypical and extreme cases tend to reveal more, because more of the underlying mechanism is exposed in them. And against the old worry that close, qualitative study just confirms what the researcher expected, he points out that people who do it seriously usually report the opposite: the material forced them to change their minds.

Public health has an even better example. In the early 1990s, Jerry and Monique Sternin worked on child malnutrition in Vietnamese villages with Save the Children. Instead of studying why most poor children were malnourished, they looked for the poor families whose children were healthy, then helped other families learn the specific feeding and caring practices those households already used. The approach became known as positive deviance. Writing in the BMJ, David Marsh and colleagues, including the Sternins, report that the early Vietnamese programmes were associated with a 74% reduction in severe malnutrition, and describe a later randomised trial in 12 communes where children in the programme grew better and ate more often and in larger portions than children in comparison communes. It worked because the answer was already in the community, hidden in the exceptions.

The average tells you where most people are. The exception can tell you where they could be.

Two details from the positive deviance literature are easy to skip and shouldn’t be. First, the exception was the start of the work, not the end. Elizabeth Bradley and colleagues, adapting the approach to hospital care, describe a sequence: find the exceptional performers, study them closely to generate hypotheses about what they do differently, then test those hypotheses in a larger, representative sample before spreading them. The outlier gives you a hypothesis. It doesn’t give you a finding.

Second, the evidence base is less tidy than the stories. A systematic review by Ruth Baxter and colleagues found that applications of positive deviance in healthcare were often poorly defined and of low research quality, rarely included comparison groups, and seldom involved staff or patients in the work. I find that reassuring rather than disappointing. It means the idea is sound but easy to do badly, which is true of most good research ideas.

Three kinds of contradiction worth chasing

• The person who succeeds where others fail. What did they do differently? Often it’s a workaround or a habit the product could support for everyone. Eric von Hippel’s work on lead users gives this a sharper edge: some users face needs months or years before the rest of the market does, and they can work as an early warning system for where everyone else is heading. The person with the strange spreadsheet might not be odd. They might just be early.
• The person who fails where others succeed. Sometimes they reveal an accessibility gap, a language issue, or a context the team never pictured. These cases are easy to dismiss as edge cases, but an edge case is only an edge from where the team is standing.
• The person who contradicts themselves. Says privacy matters most, then shares everything. Says they want more features, then uses two. The gap between what someone says and does is often where the real motivation sits. Privacy researchers have studied this exact gap for years under the name privacy paradox, and the results are genuinely mixed. Patricia Norberg and colleagues found people disclosed far more personal information than they said they would. But Spyros Kokolakis’s review shows that other studies challenge whether the paradox exists at all, and that a lot depends on how attitudes and behaviour are measured and in what context. That’s the useful lesson: a contradiction can be a real tension in someone’s life, or a sign that you asked the question badly. Either way, it’s telling you something.

When the exception really is noise

None of this means every odd note is a black swan. Some are just mistakes. A participant misunderstood the task. The prototype broke halfway through. Someone was being polite, or tired, or describing a different product. Treating every outlier as profound is its own kind of bias, and it can send a team chasing one loud person instead of the quiet majority.

So I try to ask a few questions before I decide what a contradiction means:

• Does it break the explanation, or just sit beside it? A note that’s merely unrelated is probably out of scope. A note that would be impossible if your theme were true is the one to chase.
• Can I see the mechanism? “She found setup easy” is a curiosity. “She found setup easy because she imported her contacts first” is a design idea.
• Is it rare, but not unique? Marsh and colleagues note that positive deviance works best when the good examples are uncommon but findable. Too common and they aren’t telling you anything new. Truly one-off and you can’t learn from them reliably. In a small study, one person can still be the start of the search for the second.
• Was the data sound? Before building a theory around one session, check the recording. Sometimes the contradiction is in the notes, not the person.

The goal isn’t to let exceptions win. It’s to make sure they get a fair hearing before they’re dismissed.

How I build it into synthesis

Small changes to the method make contradictions harder to lose:

• Add an “against” column to every theme. For each insight, list the evidence that disagrees with it. If the column is empty, look again. An insight with no counter-evidence usually means nobody looked, not that there isn’t any.
• Write themes as claims that could be wrong. “Setup” is a topic. “People abandon setup because they don’t know why we need their data” is a claim, and a claim tells you what a contradiction would look like.
• Give outliers their own session. Before closing synthesis, spend half an hour only on the notes that didn’t fit. Ask what would need to be true for each one to make sense.
• Follow up with one person. If a contradiction looks interesting, a single follow-up interview is cheap and often worth more than another five general ones.
• Treat the exception as a hypothesis. If one person’s workaround looks like the answer, test it with more people, a survey question, or product data before it goes on the roadmap.
• Report the exceptions. Include a slide or section called “What doesn’t fit”. It makes the findings more honest and often starts the best conversations.

There’s a quieter benefit too. When a team knows the exceptions will be reported, people stop smoothing them away during the session. The odd note gets written down instead of dropped, because there’s somewhere for it to go.

Patterns and contradictions need each other

I’m not arguing against finding patterns. A team needs to know what most people experience, and most product decisions should be built on it. But a pattern without its exceptions is an average, and nobody is average. The pattern tells you what’s happening. The exceptions tell you why, and where your understanding stops.

The next time you run synthesis, before you take down the wall, look at the note on its own at the edge. Ask why it’s there. It might be noise. It might be the most useful thing in the room.

References

9

1. Baxter, R., Taylor, N., Kellar, I. & Lawton, R. (2016). What methods are used to apply positive deviance within healthcare organisations? A systematic review. BMJ Quality & Safety, 25(3), 190–201. doi.org/10.1136/bmjqs-2015-004386
2. Bradley, E. H., Curry, L. A., Ramanadhan, S., Rowe, L., Nembhard, I. M. & Krumholz, H. M. (2009). Research in action: Using positive deviance to improve quality of health care. Implementation Science, 4, 25. doi.org/10.1186/1748-5908-4-25
3. Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. doi.org/10.1191/1478088706qp063oa
4. Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12(2), 219–245. doi.org/10.1177/1077800405284363
5. Kokolakis, S. (2017). Privacy attitudes and privacy behaviour: A review of current research on the privacy paradox phenomenon. Computers & Security, 64, 122–134. doi.org/10.1016/j.cose.2015.07.002
6. Marsh, D. R., Schroeder, D. G., Dearden, K. A., Sternin, J. & Sternin, M. (2004). The power of positive deviance. BMJ, 329(7475), 1177–1179. doi.org/10.1136/bmj.329.7475.1177
7. Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. doi.org/10.1037/1089-2680.2.2.175
8. Norberg, P. A., Horne, D. R. & Horne, D. A. (2007). The privacy paradox: Personal information disclosure intentions versus behaviors. Journal of Consumer Affairs, 41(1), 100–126. doi.org/10.1111/j.1745-6606.2006.00070.x
9. von Hippel, E. (1986). Lead users: A source of novel product concepts. Management Science, 32(7), 791–805. doi.org/10.1287/mnsc.32.7.791