For those teams that do place a high emphasis on capturing up-front acceptance-criteria and can relate to the problems I outlined, then moving in the direction of scenario-oriented acceptance criteria might prove a useful step towards not needing them at all.

Thanks for your comments Liz.

In the cases where the scenarios are obvious from the behaviour we’ve discussed, I’m happy to leave it at that, unless someone wants to automate it. An example might be:

“Replaced or refunded items should be added back into to stock unless faulty.”

The four scenarios you could associate with this are obvious.

However, when I ask, “Can you think of an example in which items aren’t returned to stock even though they’re not faulty?” someone might say, “Oh, do you remember that time when that mother came in to get a refund because the buggy was too small, but it had been recalled anyway? Yes, we need to check for recalls.” And now we have another scenario. *That* one, I’d want to write down – it’s not obvious, and it’s easy to forget. But I’ll write down that specific scenario, with a title of “Check for recalls”. And then someone says, “Oh, we can’t even refund earrings for hygiene reasons, unless they’re faulty!” And now we have another. So I’d write these down, but I wouldn’t bother writing down the obvious ones.

That’s why I can say that I’m into BDD, but not necessarily driven by specification by example – there *are* other ways to specify, and they’re OK! I find the word “specification” stops people questioning quite so freely as well, so I don’t tend to use it.

I’m very happy for people to have the conversations around as many scenarios as they want, and pretty much insist on it if it’s something new or unusually complex, but I think people are starting to put *too much* emphasis on capturing every single scenario, when some of them are really obvious. In those situations, I’m happy with rules being captured (as above) instead as proxies for the scenarios they generate. The scenarios will appear when and if we automate. In the meantime, the focus can fall on some of the unusual scenarios instead (which are frequently around new, differentiating and therefore highly valuable functionality). So we don’t “replace” the rules, because often we don’t even bother writing them down!

The big difference for me is that where a scenario *needs* to be captured, I’ll use specific, realistic, actual scenarios with realistic names and data, rather than capturing scenario-oriented acceptance criteria. I find it pings people’s imagination more, and lets us come up with the other five scenarios that we *also* forgot about.

I have done the kind of thing you’re talking about before, capturing all the scenarios to use as acceptance criteria. I found it useful in low-discipline environments, where the developers sometimes forget to check what’s working and what isn’t. I’ve got away from that recently by teaching Feature Injection instead, so that everyone has a clear idea of the value of what they’re doing and the risks associated with it, and is motivated to understand their work better.

I’ve also done it when the scenarios have been discussed during planning meetings, and needed to be captured because the people doing the work weren’t always the same as the ones doing the capturing, or enough time passed for them to forget. Persuading people to step away from fine-grained estimates (also using Feature Injection) and accept some discovery in the process of software development means we no longer have to do that either, and can talk through scenarios as and when it’s needed, thereby keeping the knowledge fresh.

As a result I suspect that I have a slightly looser process – and I don’t think either of our approaches is necessarily wrong. We may just work in different contexts.

Have a chat with Dan North about his “Ginger Cake” pattern some time, and please invite me for the conversation if I’m around 🙂

Some of the things you’ve said don’t make sense to me so either I’ve not understood you, you’ve not understood me, or a little bit of both.

Hopefully we can do something about that… What I think I’m saying, in relation to what you’re saying is:

* Your “acceptance criteria” are what I call “rules-oriented acceptance criteria”.
* I find that you can also take a “scenario-oriented” approach to acceptance criteria and I favour that over “rules-oriented acceptance criteria”.
* Your “scenario titles” are also my “scenario titles” but they can be the same as my “scenario-oriented acceptance criteria” but they can’t be the same as the rules-oriented acceptance criteria.
* Your “scenarios” are also what I call “scenarios” and I am in general agreement as to what goes into them (although our writing styles may be a little different).

In more detail…

Hi Antony, this makes sense. I think the only difference is that I’m calling the examples at the bottom, where you actually have specific albums and people, “scenarios”, where as I’d call the short-hand form for the scenarios you identified “acceptance criteria phrased in scenario form”. It isn’t until we get “visible to me, not to others” that the scenarios start to emerge, and probably if we were talking through them we’d find more (eg: what happens if Jane visits a page where Fred used to have a public album?)

I can see why you might think that calling the “given-when-then” parts scenarios is different to what I was saying since I’ve re-read the post and it’s not clear that I too also call those scenarios… Although I also acknowledged that some people also refer to them acceptance tests, so that’s not different to what I was trying to say.

How is calling the “scenario-oriented acceptance criteria” (the phrase I used) different to the phrase you’ve used:  “acceptance criteria phrased in scenario form”?

A better way of getting my point across might be that I call these “scenario-oriented acceptance criteria”:

* Can create new private album (visible to me, not to others)
* Can make public album private (visible to me, not to others)
* Can make private album public (visible to me, & others)

And I call things like this scenarios:

Scenario: Can Create new private album 
When I create a new private album called “Weekend in Brighton” 
Then I should be able to see the album And others should not be able to see it

And, by using scenario oriented acceptance criteria, we can use the criteria as the scenario titles and the scenarios become an elaboration of the criteria, rather than mis-aligned derivatives of the ‘rules-oriented acceptance criteria”.

I like to retain the acceptance criteria in some form, but often only as the title for a scenario or a short description of what we’re trying to achieve in the scenario blurb. I’ve found this useful in the past because we often miss scenarios (despite all the conversation, we don’t know what we don’t know). 

If a team found it valuable to retain the ‘rules’ as notes or additional reference material, I’m ok with that… but I’d not want them to be regarded as acceptance criteria. 

In order to demonstrate that the rules-oriented acceptance criteria in my “private albums” illustration have been met you’d need all three scenarios anyway…  All the rules in my illustration apply in each scenario (i.e. the rules are cross-cutting) and are demonstrated by all three scenarios passing… so I’m not sure what value keeping the rules-oriented acceptance criteria would add.
Can you demonstrate how the acceptance criteria (as you describe them) or “rules-oriented acceptance criteria” as I describe them, can be used as scenario titles?

I’m not completely drawn in by Specification by Example as a result…

I am confused as to how you can not be drawn in by “Specification by Example” since you’re such a strong advocate of Behaviour Driven Development… 

Unless you have a completely different idea of either “Specification by Example” or “Behaviour Driven Development” to me…

What I understand from “Specification by Example” is that it’s a general term to describe approaches that use examples, rather than rules, as the basis of a specification. Examples can be any form of stimuli with an associated outcome. I also understand it to imply that we infer the implementation both incrementally and iteratively from the examples and test our understanding by being able to execute the examples against the implementation.

Test-Driven Development and Behaviour Driven Development are forms of “Specification by Example” since we specify things by means of input-output examples (or stimuli-outcome examples). An input-output example may be very specific or more abstract, depending on what aspects of the input or stimuli is relevant to the scenario. From each example we infer the next evolution of the implemented algorithms to support these examples.

…though I do very much like talking about behaviour and having examples of it. Using the desired behaviour (or “acceptance criteria”) as a title is a really lightweight way of getting the traceability, etc., that you mention.

I think what I’m trying to say is that by reframing our rules-oriented acceptance criteria as scenario-oriented we negate the traceability concern. I wouldn’t call it lightweight… maybe zero-weight because we demote the rules to no longer exist or into nothing more than notes – so there’s not need for traceability.

My post was mostly aimed at the opposite problem – when people were specifying scenarios too early and in too much detail, resulting in glazed eyes and boredom.

That’s not the opposite problem. It’s a related problem solved in a different way. My solution to that problem is expressed in the blog post as it’s ok to discuss the criteria in terms of rules but (and this next part is where our approaches differ) replace those “rules-oriented acceptance criteria” with “scenario-oriented acceptance criteria” at the earliest opportunity. The key-word there being “replace”. In the context of this problem, I’m only talking about going as far as writing the “scenario-oriented acceptance criteria” not the full scenario.

As for the examples you gave… the scenarios are exactly the same but for some word-smithing. The final elaborated example you give is a good one… and similar to where I might end up if that was what we wanted to happen. In my example, we had decided in the conversation that the person would get a 404 when attempting to view someone else’ private album and that the scenario was adequate as it was.

I hope that clarifies what I was trying to communicate a little more.

I like to retain the acceptance criteria in some form, but often only as the title for a scenario or a short description of what we’re trying to achieve in the scenario blurb. I’ve found this useful in the past because we often miss scenarios (despite all the conversation, we don’t know what we don’t know). I’m not completely drawn in by Specification by Example as a result, though I do very much like talking about behaviour and having examples of it. Using the desired behaviour (or “acceptance criteria”) as a title is a really lightweight way of getting the traceability, etc., that you mention.

My post was mostly aimed at the opposite problem – when people were specifying scenarios too early and in too much detail, resulting in glazed eyes and boredom. I reckon it’s probably OK in that context to capture the acceptance criteria and just talk through scenarios until they’re automated, while also capturing any unusual edge-cases that come up. So in that situation I’d have:

    Make a private album public, 
    Make a new private album, 
    Make a public album private – Given Fred has a public album, when Fred makes the album private and Jane goes to look at it, then Jane should see “This album has been made private”.

I think we both agree that whatever helps people talk, in their context, is fantastic. It’s been a while since I’ve worked on a project where the testers haven’t *wanted* to be more involved, so thank you for reminding me that this happens.