Confirmation bias is a really big problem that stalks us when we attempt to test things, making us complacent about how good a job we’re doing. I hadn’t realised how much of a problem it is until recently.
When playing testing games with friends and peers, and when teaching new grads about testing, I often use the dice game and the 2,4,6 Wason Rule Discovery problem. (The youtube link embedded in the linked page covers the problem and confirmation bias better than I’d summarise here – if you don’t know about the problem, go and watch that.
The Rule Discovery problem in particular is a really good example of how confirmation bias affects us and gets in the way of testing. Although subjects given this task typically expressed high confidence in their guesses, only ~20% of the subjects in Wason’s original experiment successfully guessed the real rule, and replications since then have continued to show success rates of around 20% .
But I now think that the issue is worse than I thought…
When trying these games with my peers, one of the issues I regularly see is people (including myself) hampering themselves by being reluctant to be wrong or appear “stupid” in front of their peers. And I’d always somewhat assumed that that was part of what drives/conflates with the symptoms we see that apparently involve confirmation bias.
About a month ago, however, I was talking to some year 9s about STEM (Science, Technology, Engineering, Maths) careers. As part of this, I played around with a couple of puzzles, including the dice game, and the 2,4,6 problem. The year 9s were very bright and were also much less constrained with their ideas than most people I’d come across, and totally unashamed about “silly” ideas or being wrong. However, they still had the same positive gluing onto trying to prove the rules they came up with rather than disproving them.
Which is a shame, because I’m reasonably good at noticing when I don’t want to embarrass myself, but it still takes me conscious effort to force myself to look for ways to prove that I’m wrong.