Interrupting the normal schedule, here’s a thing that I found interesting (long post warning):



Games for Testers: Sherlock Holmes Consulting Detective


So I’m into board games (no – not monopoly, real boardgames) and this is my current favourite.  Actually anyone with a sciencey investigative mindset will love this game.  From now on, anyone who asks what could be fun or interesting about testing is likely to get a copy of this thrust in their face as this really embodies the fun of poking around and investigating a mystery until you understand it.

In short, you and your friends take on the role of the Baker Street Irregulars.  Holmes introduces you to a new case with some initial thoughts and leads, and then you’re on your own to explore London and track down information from a huge massive choose-your-own-adventure-style book.  Except there’s no “turn to page X or Y”.  You can go anywhere.  Where do you want to go?  What’s useful?  That’s for you to decide.  And you only stop when you want to.  At which point you go back to Holmes Study, answer a quiz (which you don’t get to see before you declare that you’re done) on what’s been going on  and then Holmes grandstands and explains the actual answer and how he worked it out (inevitably far faster than you – but the information was all there for you to see).

You win and lose as a group, and while you can play it alone, a group of 4-5 is great, because the game rewards knowledge, thinking, understanding and the sum of investigators thinking and riffing off each other is vastly better than the individuals.

For a longer and better (and slightly surreal in places) review of the guts of the game, try this.  It’s worth a watch.  Go on.  I’ll wait.

But this isn’t just a great game.  The things that make it fun are exactly the things at the core of testing.

  • At any point, you have control over what you do, where you go, what you decide to investigate, and you make the judgement of when you’re done understanding the world.  More (correct) understanding means more points at the end of the game.  But each extra test costs you a few points.  So the key mechanic is the judgement of when to stop, when to search wide, when to search deep.
  • It rewards diverse approaches, thinking laterally, and the ability to quickly create and abandon many hypotheses.
  • You’ll inevitably make a model of how the game works and make assumptions based on that – and then discover your model was broken.
  • Sometimes the problem is that you don’t know what you don’t know.  Sometimes you know exactly what you don’t know and the cleverness required is in constructing a way to find it.
  • While there’s one main mystery to be solved in each case, you can also uncover a whole load of other things (which get you more points).  There’s an unknown number of things to find – and some of them aren’t worth the time and effort required to find them.


If you buy this (hint: you should), here’s one steer that would have made our first case much better:  Just like in testing, going to too few places and getting the wrong answer is much worse than doing more digging and getting a right-er one.  Typically, investigating finds other interesting things (with bonus points) and the game has a lot of detail to enjoy, so its definitely worth going to more rather than fewer places.

Now go and have some fun.

Books: Lean In and Lean Out



I’ve just finished a couple of books and accompanying criticism.  Given the context of a recent events, it seems as good a point as any to write about them.  “Lean In” and “Lean Out” (actually Lean Out is a collection of essays).  Both are about dealing with sexism in tech.

Lean In, I found interesting.  A bunch of thoughts and examples where women miss out not through someone being explicitly openly sexist, but through a load of continual low-grade things that might bunch go under your radar where “things people do that we recognise as good development” tends to align much better with “things men do” than “things women do”.  A load of examples added to my mental list of  things to actively look for to help people (and also incidentally a bunch of things to look out for, to help anyone who has lower confidence).  Lean In isn’t a magic fix, but is definitely a useful read for anyone trying to work on the problem “The tech company where I work seems basically OK on gender, but I know about subconscious bias; I wonder what we can and should be doing better?”

Lean Out, was much more sporadically useful – which is reasonable to expect from a collection of essays.  The biggest thing I took away from it was that my prior assumption that most of tech is like where I work is apparently flawed…

I was reading these books in the context of worrying about improving a culture at work where we’re rationalist, try to treat everyone the same, but social norms and subconscious bias work against that.  “Lean in” provides good thoughts that help on that, working against the biases and over time creating a culture where those biases are fewer or don’t exist.

It seems that that’s not “normal” in at least a significant fraction of tech companies (from  what I’ve read it seems that this is especially true start-ups or companies that have grown fast from start-ups and not had time/effort put in to develop their culture).  In a company where the culture is be a bunch of total arseholes as described in several of the essays in Lean Out and in wider news recently, I can see and feel why it seems reasonable to be angry and shouty and go down the “don’t fix it – bail and start again” suggested by various of the Lean Out essays.  For the record, it shouldn’t be for “feminists” to hold arseholes to account.  They need to be held to account by any humanist.  It’s not (just) about not being sexist.  It’s about not being a dick.

That’s not to say, I’ve stopped worrying about how to improve our company culture (Just because we’re miles over the “Not being full of arseholes” bar, doesn’t mean we should get complacent!)  More, that Lean Out in particular helped me see and understand where some of the anger is coming from.




The British House of Lords as a Testing Organisation

Lords.  A British institution involved in Tests.

The British system of government is highly anachronistic, and nearly unique.  This post will cover a brief summary of the system, what’s unusual, and the surprising benefit that it gives.

How does the British Parliament[1] work to produce legislation?

There are two houses, the House of Commons (HOC) and the House of Lords (HL).

The UK has no “constitution” per se.  It has a body of laws and on top of that various parliamentary conventions (for example HL doesn’t usually get involved in budgetary legislation).

HOC creates all legislation.  If the legislation is passed (majority vote) within HOC,  it is then submitted to HL.  HL reviews the legislation and either passes it or kicks it back to HOC – typically with details of what they don’t like and why and suggestions for ways to make it better next time.

Two other side-notes on the process.

  • HOC isn’t required to make all (or any) of the changes that are suggested, and HL can keep rejecting legislation with markups, so in theory HOC/HL can end up at loggerheads.  However by convention HL pass any legislation third time round and in practice the loggerheads never happens[2].
  • Once passed by HL, the legislation is then submitted to the reigning monarch who can also veto anything.  Again to my knowledge this has never actually happened.

So, who makes up HOC and HL?

HOC consists of 650 MPs, who are voted for in general elections every 5 years.  It’s your normal, standard “vote for someone to represent your area” style democracy.

HL consists of around 825 people, who are a slightly odd crew, but mainly composed of people thought (or thought in the past) to be good at working out what’s good for the country.  The vast majority of them are “Life Peers” (700 or so out of 825) – they’re people from all over who’ve been added to HL for life as people with something useful to contribute.

So, we’ve got HOC trying to build stuff to their best interpretation of what the customers (the population) say they want, and HL, a separate organisation trying to pick apart the new stuff, spot the holes, find the unintended consequences, and where the interpretation of what the customer wants probably isn’t what the customer actually wants or needs.  And then passing back a bunch of bug reports and suggestions for making things better.  And steps have been taken to make sure that (a) the HL team can’t be directly pressured by HOC (separate chambers) or the customer (no re-election) and (b) they provide information as a service to the HOC team, rather than owning the release schedule or having control over what exact product is shipped and when.  That sounds like an organisation with a strong independent test team.  Huh.

So a final random political thought here at the end.  There’s some debate in the UK about how to reform or abolish HL.  I’ll not weigh in on that debate, but as I see it, the key benefit we get from the HL is their role as an independent test organisation – which is enabled by having a bunch of people who by their decisions cannot affect their re-election – thus freeing them to make decisions for the long term benefit of the country as they see it[3].  The interesting thing to my mind is how you go about carefully choosing people to join that group and how exactly the interactions work between the expert testers and the dev team to ensure that the customer ends up with a good product that they’re hapy with short and long term!





[1] I’ve used the horrible Americanism “British” to label the process of lawmaking in Westminster.  In doing so, I’ve glossed over the various national assemblies, the excitement and confusion involved with Britain, The United Kingdom, Wales, etc.  For a fun diversion into the start of just how complicated this gets, check here for an explanation of the various countries that make up the UK.

[2] There are a couple of other conventions about what HL does and does not comment on and when they step out of the way.  I’m not going to cover details here as it’s complicated and I’m sure I don’t know all of them – and it doesn’t really affect the point I’m trying to make in this post.

[3] An example of what happens if you don’t have that independence:  The US houses of Senate and Congress (modelled on the original British system but made more democratic – more voting is good, right?) manages to add all the complication of an additional house while wrecking the main benefit from the British system by requiring re-election of their testers every few years).

Testing thoughts from the back of a Yoga class

The closest I could find to a picture of me doing Yoga

I do yoga, as an enthusiastic amateur.  The main weekly class that I go to can best be described as “the basics, really well”.  Think of it as the equivalent of a concert pianist practicing their scales – basic stuff for them but the foundation of a good technique. Just doing scales and arpeggios isn’t going to make you a concert pianist, but in order to be one, you’d better be prepared to understand and be really good at your scales and arpeggios.  So the yoga class is an open class with a mix of newbies and people who are very very good, but with a focus on doing the basics right.  The teacher, Nadia Narain, is excellent, especially for a geek like me; for the asana practice (the physical poses part of yoga) she puts a lot of effort into explaining exactly what’s going on with your body in each pose and what the core point of a pose is, rather than just telling the class what position to put their arms and legs in (effectively talking about the requirements behind the pose, not the specification).

It took me a couple of years to realise that a lot of the adjustments that Nadia and her assistants are actually doing on the good people in the class are actually pulling people out of deep poses – to stop them from enthusiastically (almost competitively) doing what they think the pose is about, not what it’s actually about.

A simple example – Janusirsasana (Head to Knee Pose) is a twist and stretch pose, tightening various core muscles, aligning and tilting your pelvis, stretching your hamstring and creating space in your back. If you’ve been doing a lot of yoga, for a long time, you might end up with your head in the vicinity of your knee – hence the name.  Or once you’ve done a bit of flexibility work, you can curl around like a letter “d”on it’s side and whack your head on your knee, stretching your hamstring, but missing the core points of the pose and stretching and wrecking your upper back into the bargain, while performing a literal head-to-knee pose.

And the thing I eventually realised here is that these aren’t people who are bad at yoga, or have bad teachers – these are good students (most of them far above where I’ll ever be) who are trying to do good work.  Instead, it’s a weekly reminder of some everyday truisms that apply all over, that we often forget, and that are especially important to remember when testing:

  • What everyone sees and talks about isn’t necessarily the interesting thing that’s going on.
  • What appears to be the obvious point or rationale of something isn’t necessarily what’s going on.
  • Everyone (customers, users, developers, testers, whoever) will all enthusiastically do what they think they’re meant to be doing without asking.

So next time you’re deep into some testing, don’t forget to occasionally back out of the pose a little and ask yourself what you’re actually trying to achieve.




Testing and Cooking

A tester

This post was triggered by thoughts following on from a Michael Bolton tweet and blog post making analogies to explain that test cases are not tests.

The analogy (a test case is like a recipe, and the testing you’re doing is the cooking) is interesting in that it helped me realise why people often love test cases so much.


As a student, I used to cook without recipes.  And the food we made was.. edible.  And we didn’t know better (and besides we didn’t have much to spend on the ingredients).  Then as a graduate in my first job, being paid real money and everything, I discovered (a) more complex ingredients and (b) cookery books.  For geeks.  With detailed recipes and everything.  And I loved it.  My results when cooking were much tastier, and crucially I discovered the more detailed and precise the recipe, the better the result.   I craved detail and precision – and my results were better for it.

My wife is an excellent cook.  She knows what she’s doing, how flavours work together, and her sense of taste is exceptional.   When she cooks from recipes, she doesn’t care about the great detail.  She’ll start on the recipe – and then substitute half the ingredients, swap all the timings around and end up with something technically different, but that tastes way better.

I can’t replicate what she does, even when I used to write down all the tweaks she made, because what she’s doing is cooking – reacting and working with what’s happening in real time – nailing that down and turning what she did into a detailed recipe and repeating that makes things worse – because so many other things are going on that aren’t written down.


And that, I think, mirrors into testing.  CDT testers have developed an allergic reaction to test cases (detailed recipes), and that’s a good discussion to have and way to think for people who are already basically competent cooks and can be freed from the recipes into thinking for themselves.  (Now I’ve got some reasonable cookery skills, my cooking has improved significantly since I’ve stopped worrying so much about the details of the recipe.)  However, for people starting out, the reverse is true.  Following a nicely well constrained test case gives demonstratably better results than them fumbling around in the darkness, and so they react badly to people telling them that contrary to their personal experience, the reverse is true.  Instead of telling them “recipes are bad”, we need to tell them that recipes are ok to enable them to start their journey to become a good cook – but that following a recipe by rote does not make them a cook, or improve their cooking skills.  We also need to help teach them the skills of cooking and encourage them to explore around their recipes and onwards, so they can take pointers from recipes and do their own thing.



Book: Turn the Ship Around

I’d say “No ships were harmed in the making of this book”, but I’m not actually sure if that’s true.

This is a book about management (we read it in our management book club at work), rather than testing.  I found it first pulled out clearly a lot of vague thoughts I’d been having about how to develop bright people (or more truthfully enable them to develop themselves), and ways to ancourage people to safely pull delegation of responsibility in an organisation with quite strong management lines and a strong reputation and focus for delivery.  It then blew me a way as a demonstration of how much farther you can go than I’d been thinking.  It’s part biography of life on a submarine – which helps keep you reading – and part briefing on the take home lessons.

One of the things I liked most about it was that it is the author‘s description of a journey, and what he and others tried and didn’t try, why they did so, what did and didn’t work, and what they learned at each step – rather than a presentation of “Here’s the answer we got to, listen to me, I’m amazing.”, which many management books seem to be.

If like me, you’re trying to balance a passion for people development within an organisation with a strong reputation and focus for delivery, I thoroughly recommend this book as a demonstration and tool to let you deliver on both without compromise.