2-4-6 – Humans are instinctively poor at testing

4-6-2 does not fit the pattern

Confirmation bias is a really big problem that stalks us when we attempt to test things, making us complacent about how good a job we’re doing.  I hadn’t realised how much of a problem it is until recently.

When playing testing games with friends and peers, and when teaching new grads about testing, I often use the dice game and the 2,4,6 Wason Rule Discovery problem.  (The youtube link embedded in the linked page covers the problem and confirmation bias better than I’d summarise here – if you don’t know about the problem, go and watch that.

The Rule Discovery problem in particular is a really good example of how confirmation bias affects us and gets in the way of testing.   Although subjects given this task typically expressed high confidence in their guesses, only ~20% of the subjects in Wason’s original experiment successfully guessed the real rule, and replications since then have continued to show success rates of around 20% [citation needed].

But I now think that the issue is worse than I thought…

When trying these games with my peers, one of the issues I regularly see is people (including myself) hampering themselves by being reluctant to be wrong or appear “stupid” in front of their peers.  And I’d always somewhat assumed that that was part of what drives/conflates with the symptoms we see that apparently involve confirmation bias.

About a month ago, however,  I was talking to some year 9s about STEM (Science, Technology, Engineering, Maths) careers.  As part of this, I played around with a couple of puzzles, including the dice game, and the 2,4,6 problem.  The year 9s were very bright and were also much less constrained with their ideas than most people I’d come across, and totally unashamed about “silly” ideas or being wrong.  However, they still had the same positive gluing onto trying to prove the rules they came up with rather than disproving them.

Which is a shame, because I’m reasonably good at noticing when I don’t want to embarrass myself, but it still takes me conscious effort to force myself to look for ways to prove that I’m wrong.







Ask A Manager

Really handy blog

So when I started writing this blog, I envisaged that sometimes interesting questions would come up regarding people management, which I could write about.  And they do (people are way more interesting than computers).  However, most of the “this happened and I learned a thing” thoughts I have aren’t easy to safely anonymise – especially given the pretty small readership of this blog is mostly from my place of work (Hi folks!)

So instead, here’s the general advice I try and live by, and try and encourage in others.  It’s pretty simple, and is a good starting point for managers, managees and coworkers trying to cover any kind of problem or issue.

  1. Talk about the issue to people who can actually help close the issue down.  The number of issues I’ve encountered where “Bitch all about this issue in the pub or on one of our chat channels” was considered an acceptable replacement for dealing with it really surprised and shocked me, until I realised that I used to do exactly that.
  2. Initially assume good faith.  Differing viewpoint >> Partial Ignorance or Partial Incompetence >> Malice.  Even if you already think you know where the other person is,  be ready to discover that they’re back towards the “not-a-bad-person” end of the list.
  3. Bring your adult ego state to the conversation, and engage their adult.  If they bring their parent or child, then you can engage with that, but be aware that for any interesting issue, you won’t be able to close it out until your adults have engaged and accepted the next steps.  If this doesn’t make sense, read the first few chapters of Games People Play by Eric Berne.  Maybe someday I’ll try and post a summary, but I’ve tried and given up before.

I’ve said the advice above is simple.  That doesn’t necessarily mean it’s easy.  In particular, there are two problems that I keep bumping up against (with myself as much as with others).  Over the years I’ve converged on two blogs, which I recommend as they help with these.

  1. Being honest and adult.  Our brains tend to make stories that paint us as heroes, and we don’t like to see anything against that – which means having grown up conversations, assuming that the other person isn’t a total arsehole, maintaining your cool and adult engagement, etc. is really damn hard.  Ask a Manager is full of useful advice for how to approach issues sensibly and pitfalls to avoid, and if you look on there, probably the one you have (or a similar one) is covered already – probably from both sides!
  2. Talking to other people.  Especially where I work (software company full of geeks), actually talking to people, about issues, that involve conversations that might have tension in, can be daunting.  For this, I’ve found Captain Awkward useful.  It’s not specifically work related (though the “Work” section is large), but it contains lots of conversational starters – Captain Awkward calls them “scripts” – which are good ways to help break that initial barrier.


=== Edit 2017/10/08 – Fixed Ask A Manager link

Developing your testing using jokes

Why did the developer cross the road?  To get to the others’ ide.

Another in a somewhat sporadic series of “things you can do in your spare time that are fun and hone some of your testing skills.  This one comes from a session a year or two back at the Assurance Leadership Forum (formerly Test Management Forum) by James Thomas.

Joke creation flexes just those same skills you need for testing.  Logic, reasoning and deduction, lateral thinking, breaking assumptions, and that intuitive flare for spotting the edge of something that niggles and pulling at it until it unravels.

And it turns out, that James has now released an eBook which you can read here (don’t panic – it’s a 20 page PDF) that talks through and unpicks some of this far better than I could explain.  I recommend the read, and then giving creating some Jokes a go!

“Get Stuff Done” time



So I’ve found over time that while I’m pretty good at mopping up the easy tasks, and I’m pretty good at doing the fun tasks, I’m fine with doing the urgent tasks (even the un-fun ones), and the quick tasks are no problem either.  I’m even “OK” at doing the larger hard tasks.  But man oh man does my task list slowly fill up with the not-super-urgent-and-not-super-quick-low-grade-crappy tasks that only take 30 mins or so but I don’t want to do.

The only technique I’ve really found to get these done is a “Get it done” session.  I line up my backlog of tasks, get out my phone and a tally-pad – and see how many I can knock off in a pre-defined time.  Typically when I do one of these, two things happen.

  • I manage to get a load more items knocked off than I thought I could.  Turns out when I’m putting the effort into pushing myself rather than hating the work, I can get loads done.
  • The 2 hours rolls around so fast and I feel really productive and good afterwards. (Sorry colleagues who sit behind me and occasionally have to put up with me getting all excited and triumphal.)

I always kind of assumed that this was because I was a bit weird and childish, and then I read this blog post, which pretty much covers the same idea (on a larger scale).  So it’s not just me being weird and childish.   Work tasks, chores at home, whatever, I’d really recommend giving this a go.



How to win at RTS games

Enter a caption

Real Time Strategy (RTS) games typically have have a couple of different resources that you collect in some way, and you use those to build a bunch of buildings that let you make units to that let you go over the other side of the map and crush the other person.

Anyone used to DOTAs, FPSs, other games will pretty quickly pick up that different units have different useful abilities and that if you practice with them and use your units together, you’ll do better at fights, and beat other people’s army.  You’ll practice and beat some people who haven’t done as much practice as you.

And then you’ll start getting absolutely crushed by people who just built a big shed-load of stuff and told it all to go kill you.  Because until you’re in the top 10-15% of players of a particular game, you’ll get much better returns putting your effort into getting really efficient at creating your buildings and army and ending up with an army twice as big, than you will in squeezing a little extra damage out of each attack.   In RTS terms we talk about Macro (getting your economy going and building stuff – the big strategic stuff) and Micro (getting the most out of your individual units). And until you’re very, very good and can manage to do both very well, Macro >> Micro.

So what’s all this rambly stuff about computer games doing in a blog about testing and management?

Well the reason for the Macro vs Micro question at all is because there’s a hidden secret resource in all RTS games – the same hidden resource mechanic for any project that you’re managing – your time and focus.   And that same choice to focus on Micro vs Macro (and the same relative benefits) apply to projects too.

If you train yourself to put all your time and effort into adding a little bit of extra benefit to helping the people on your team do their jobs better, then you’re focusing on improving Micro.  (That’s not that you’re Micro-managing – but you’re focussing on individual tactical details.)  Your team will do better, and they and you will see it and feel better, because you’re delivering more.  Hurrah.

But you could be doing vastly better if you’re putting your time an energy into the Macro, letting everyone get on with their jobs – even when you could spend your time and effort coaching them a little bit extra – and instead focusing on making everything surrounding those jobs smooth and efficient.

In reality, we want to do both Micro and Macro, but if you’re not providing both, then Macro is probably a more important place to spend your time.


Voting. Back your kickstarter to make the country better for you.


So this post isn’t about testing or management.  It comes off the back of a few conversations with some late-teens about whether its worth bothering to vote.  Now I’m old and grumpy and definitely not cool (in fact I’m pretty sure that “cool” is not cool these days).  But there’s a snap general election happening in the UK, so if you’re going to spend a few minutes thinking about whether to vote or not, now is a good time – so I figure I’ll float some thoughts.

(Spoilers – I think that you voting is a good thing for you, but don’t do it because I say so, make your own mind up[1].)

As explained to me, there are two big reasons not to vote.

  • None of the options are any good.
  • My vote won’t make a difference anyway.

I’m actually going to poke at these in reverse order, because why not.

Lets look at kickstarter projects.  Let’s say there’s a kickstarter project up for a new card game about exploding kittens.  Your own personal pledge backing it is unlikely to make a difference as to whether it gets off the ground.  But if you think it’s worth it, and your mates do, then likely their mates will and so on, and if you all pitch in, then suddenly, BOOM there’s actually a game there out of nowhere.

Or not.  Maybe despite your best efforts, there just aren’t enough other people out there excited about kitten explosions to get to the funding needed, so nothing happens.  The beauty of kickstarter is that you’ve lost nothing – well technically you’ve lost the time you spent signing up and the time you spent reading about kitten explosion techniques and the time it took you to make your pledge.  But that’s it.

And interestingly, if a bunch of people show that they’re interested in card games that involve explosions, then people will think “that’s interesting – people like explosions in card games, and before too long there’ll be another one along.


Voting is pretty much like that.  Politicians create their campaigns (or kick starter projects in the above analogy) to try and coax people to choose their project.  And they do that by looking at which projects people funded in the past, and then trying to pitch a project that sounds like that.

So vote.  If you get a groundswell of support – then you can make any campaign deliver.  Even if none of the mainstream candidates looks useful or like they’ll make a difference – find a candidate or party that matches up with roughly what you’d like to see.  Even if they don’t get in, the mainstream candidates will notice and try and find out what you’re interested in.   And so next time round there’ll be better looking choices[2].

And if you don’t vote.  Then those politicians setting up their campaigns next time, will be setting out all the juicy pledge bonuses for the people who do seem to care.  Because whether or not they actually care about you,  if you don’t vote, they don’t have to care about you, and they’ll be putting their efforts into someone else.


[1] In fact technically you not voting is good for me, unless you’re just like me and care about the same things I care about.  Life would be much easier (for me) if no one except me voted.

[2]Hopefully.  I’ll be honest, I’ve been old enough to vote in 5 general elections, and while in general this seems to roughly happen, sometimes it does seem like everything is just an absolute shambles.