Simple Design and Testing Conference

The Simple Design and Testing conference consists of two days of Open Space discussions between a small group, primarily developers with a keen interest in Agile. It kicked off in London on March 12th with lightning talks for introductions, before lining up the Open Space sessions.

On Day 2, Liz Keogh held two sessions that were closer to presentations than discussions: Feature Injection and The Evil Hat. Both were excellent, and it was quite a nice contrast to the open space sessions to have a couple that were more structured.

Tell Don’t Ask

Tom Crayford kicked off the session by asking what if all methods returned void? There was a lot of discussion about the patterns followed in the “GOOS” book before the group moved on to talking more about functional programming. Although I didn’t contribute much the discussion was interesting – I’d love to try a coding kata one day with all methods returning void, it sounds like an interesting challenge and I’d like to see how it turns out :)

Writing Tests against a Legacy Codebase is a Good Thing

I was looking for good ways to convince people that doing testing – unit and acceptance tests – against a legacy codebase is a Good Thing. I thoroughly enjoyed the discussion and plan to post more about it later, but what I found most interesting was the difference of opinion even within the group. Coming around to really, truly seeing and believing in the value of writing and maintaining automated tests can be a very long process.

Speculative Coding

Many teams and organizations still build software a layer at a time, with teams divided along technology and architectural lines. Entire APIs are built before the client that will consume them is started, with work being wasted because we speculate as to what will ultimately be needed. Finding ways to get fast feedback such as attacking the most complex and interesting scenarios first can help to avoid this waste.

Wicked Problems

I’m still a bit of a newbie when it comes to Systems Thinking, and I hadn’t heard of Wicked Problems. The definition is that this is a problem that doesn’t have a solution. You don’t know when you’re done, and you can’t experiment – every possible solution is a one-shot operation. Like testing software. You never know the best ways to test the software until you’ve done it. Liz pointed out that the same could be said of analysis. It was an interesting way of looking at things, and I definitely left the session with even more respect for testers.

Simplicity Debt

If we try to keep things simple, do we just defer complexity to a later date? Complex problems usually require complex solutions. We quickly realized that we didn’t really have a common understanding of “Simple”. Does it mean the least amount of effort, or the most elegant solution? Is simple really the opposite of complex, or do we actually mean the opposite of compound? I think we all came away from that session thinking wow, simplicity really is quite complex.

Feature Injection

Liz explains feature injection as a pull system to derive the requirements for the features we want by starting with the vision for a project. To meet the vision, we must achieve goals, which require the system to have certain capabilities. Features work together to form capabilities, and features are further decomposed into stories. Liz suggested that stories having true business value in their own right is often a myth – the main value of stories is that they can be completed on their own and provide a means to get feedback as fast as possible. Most stories can’t be shipped alone.

The other really useful insight from this session was that we often talk about stories from the point of view of the user, but in actual fact most of them are not for the user’s benefit. Does the user really want to see adverts on the page? If we think about the story from the stakeholder’s point of view, it’s far easier to articulate the goal – As a stakeholder, I want adverts on the page, so that when users look at the page I make money.

Why do we estimate?

Estimates, Daniel Temme proposes, are lies. There’s no value in lies, so why do we persist in trying to estimate all of our work? Why can’t we just put the work on the board and pull it in, and it’s done when it’s done? It’d probably be done faster if we weren’t wasting time estimating (and complaining about it). There are companies out there like Forward and DRW who are doing this successfully, blogging and talking about it. I haven’t made up my mind where I stand yet on this one.

Liz’s Evil Hat

I thoroughly enjoyed the Evil Hat talk. I think I just like hearing other people’s horror stories, and Liz has some great ones. It’s also a really interesting point of view to take, which is that most measurements and metrics are really just opportunities for people to game the system. Even if you don’t do it deliberately, that little devil is always on your shoulder, whispering in your ear.

Types of Testing

This session had been born out of a confusion in a previous one between terms for tests like functional, integration and acceptance. I think in the end, we were in general agreement that functional and acceptance tests and scenario tests are pretty much all the same thing. Integration tests usually test the interaction between two or more objects, but not the entire system. In the end, I think the most important thing is that your team have a common understanding of what you mean. Your functional test by any other name would still be green and shiny :)

Jasmine Testing Dojo

We attempted to run a jasmine testing dojo later in the afternoon, but ran into technological challenges (Liz had the only projector adapter, and she’d left for lunch …) but we managed to write a few tests and use some spies! I’d love to create a short Jasmine testing workshop to run in a user group in future, so watch this space!


We all agreed the weekend had been a great success, although for next time hopefully we can get more publicity and a slightly larger group. Finally, off home to rest my brain and have a well deserved glass of wine …

Estimatation …

This post was originally published on my last blog: Spike the Poodle

Imagine you and I were looking up at two skyscrapers, and I asked you how high you thought one of them was. You'd probably find it pretty hard to tell whether it was 50m or 500m. But what if I asked you how high the second one was in relation to the first? I bet you'd be able to say it was about a third larger, for example. If you could measure the height of the first building, it'd be much easier then to estimate the height of the second.

People are generally better at estimating relative size than absolute – and this doesn't just apply to buildings. In software development, estimating relative size – usually with story points – and then measuring how many points we can deliver in a given time is generally more accurate than trying to figure out how long a task will take in days, and then allowing for the general daily faffing.

Most developers have spent their careers estimating in time though, and making the transition to story points isn't always easy. There are a few things I've seen recently that can help people get comfortable more quickly with story points.

1. Estimate in a group with developers who have experience with story points.

Having developers in the group who have done it before is probably the best way to get the developers new to story points up and running quickly. They can set a good example and reassure the others that the technique does work. In an inexperienced group, doubt can plague the group and undermine the estimates.

2. You don't have to estimate QA and Dev effort together.

Some agile teams try to estimate the effort to complete a story to include all of the dev and QA work, but this is really, really hard. QA and development are very different skillsets, and it can be frustrating for the team members to have to estimate something they can't be expected to understand.

For teams practicing test driven development and writing unit tests, QA involvement can happen at the start and end of story development, with the QA and BA working together to write acceptance tests prior to development and sign off the story when development is complete. Unless the QA effort is really significant, it's probably OK to represent the dev effort in story points, and have the QAs check the estimates, and just highlight any that are unusually low but might cause a bottle neck for them.

3. Agree a good scale.

There are two common scales for story points: the fibonacci sequence and the power of two, and both work well. The important thing is to agree a sequence in advance, and ensure the upper bound isn't too high. Either 1, 2, 3, 5, 8 or 1, 2, 4, 8 works well. You might choose to use 0 and 0.5 for really small things like configuration. 13 or 16 point estimates become less meaningful and far less accurate – can you really think in terms of 13 times the size of a 1 point story?

Give developers a question mark for the stories they can't estimate – these are the ones that are too big, or have too many open questions. They'll need more analysis or splitting before you can estimate.

4. Get good story granularity.

Whatever estimation scale your developers agree, your stories will need to be small enough to fit into it. Usually your largest stories should be no more than eight times as big as the smallest ones, and you'll need to be clear about what each of them really means.

The best way to judge the granularity is to have some of the more experienced developers review them before you start estimating. This doesn't have to take very long. On a recent project, we spent maybe three or four sessions with developers prior to estimation, each one about an hour long. It might sound a lot, but it's worth it – there were only two stories out of nearly 700 that they couldn't estimate.

5. Capture Low, Likely and High estimates.

In some versions of planning poker, you're supposed to aim for convergence between the team members on an estimate. You can do two or three rounds of estimates on a single story to try and get there. The trouble is that this can take a long time, and get frustrating if team members don't want to budge.

One way of getting around this is to capture low, medium and high estimates, taking the lowest, highest and middle ground from the estimates from all of the developers. They may not agree, that's OK, and it's also OK to capture the disagreement and move on.

The low and high values can be used to weight estimates for release planning.

6. Get comfortable with assumptions.

Stories are a promise for a conversation. You won't know all the details of the requirement when you estimate, it's agile, you're not supposed to. In some cases, you might have a better idea in your head than what's written down on the card.

It's OK to make an estimate based on assumptions (make sure you record the assumptions with the story though!) Assumptions can help move the estimation along faster. If it turns out the assumption was wrong, then the estimate is also wrong and can be redone later.

7. Be prepared for estimation to take a long time – especially at the start!

In a recent estimation session I attended, the first estimate took well over ten minutes of discussion to arrive at. Some people were looking at the list of stories and starting to sweat and wonder if we'd ever make it through to the end!

This is normal, especially in a newly formed team, they will take time to speed up, but they will get there. Don't try and finish estimation in an hour, it's not going to happen unless you've got a week long project. The team need an adequate amount of time to not feel stressed and get used to estimates, it will pay off when you're able to speed through fifty stories in an hour and get an accurate release plan later on.

8. Calibrate several stories in advance with a smaller group.

Calibration is the process by which you arrive at the initial size of a story point. It's time consuming and challenging – we took an hour or so to calibrate four stories. If you're working with a mixed team of developers who are both experienced with and new to story points, it's worth taking the time with those who are more comfortable with story points to do the calibration first.

It's key to pick the right stories for calibration. Whatever scale you choose, find a story that fits each bracket – you may have to go through several to get there. When you're done, pin them up on the wall and refer back to them for the first few rounds of estimation: "Is this the same as that 3 point story on the wall?"

9. Use ideal days in the initial calibration.

To start calibration, use a scale like 1 ideal day = 1 story point – but only use it for the initial calibration! Once you have your scale, don't refer back to ideal days as it will muddy the waters – it's OK to use to get going, but after that try to just size against the other stories and forget about duration.

10. Triangulate with a sample of stories.

Triangulation, or cross checking the consistency your estimates, can happen at any point during or even after estimation.

You don't have to check every single story if you have a lot. One technique is to sample 5-10 from the middle estimates (e.g. 1, 3 and 5) and compare them. If they're reasonably consistent within and across the estimates (i.e. a 3 is about three times a 1) there's no need to compare any more.

A quick conclusion …

Like many aspects of agile, estimating with story points is not easy, but it is effective. I plan to write soon about going on to use the estimates for release planning, so stay tuned!

Agile 2006 – Planning and Tracking on Agile Projects – Mike Cohn

Sunday 14:15 – 15:00


The first question on a project is always “when will you be done?” Agile planning takes place on three levels, using relative story size and previous performance (velocity) to plan iterations and releases.


Agile teams plan at the release, iteration and daily level.

Release planning describes “Here’s where I want to be” – so that we can figure out the best way to get there. Releases usually comprise multiple iterations, because stories are “done” at the end of each iteration – not necessarily cohesive. Estimates at this level are ball park numbers – it should take no longer than 3 minutes to get to the estimate for each story. During release planning, the team explore the solution space and negotiate how they can satisfy the customer’s needs.

At the iteration level, tasks are estimated in “hours I thought a lot about”. There is no negotiation on schedule or budget at this level – long iterations with varied lengths don’t work. At the end of the iteration, get feedback of what’s done and left to do.

At the daily level, the daily stand up meeting is used to plan the next day’s work and synchronize the team. This should not become a status meeting.

Humans generally are bad at estimating the duration of a task, but relative sizes are more straightforward. Some teams use t-shirt sizes to estimate stories, but it is difficult to agree on the relativity – is a large story twice the size of a small one, or three times? Story points, which are the most commonly used now, make this straightforward.

A common mistake is to consider the skill of an estimator important – it isn’t. Regardless of time, people can agree on the size of a piece of work.

To estimate velocity, use a range. When considering the long term average, look back 4-5 months and no longer, as the team’s situation will have changed too much if you go back any further. Use the velocity to look at the release plan and estimate where we’ll finish at the slowest velocity and expected velocity. These estimates can be used to reprioritize the stories.

Burndown charts show progress. Teams generally work best when they have slightly aggressive, but not unrealistic, schedules – which is why the end of the burndown chart is usually steeper. Burndown charts raise questions – they don’t answer them, but they identify areas for further discussion and commentary.

The powerpoint contains diagrams, further explanations, and photos of example taskboards.

Key points to put into practice:

  • Use velocity to estimate the relative likelihood of stories being completed in a release, and use this to drive reprioritization.
  • Use burndown charts to identify areas for discussion, where further explanation is needed.
  • Track velocity only about 4-5 months back – changes in the team’s situation can cause velocity to change over time.

How many estimates?

This week, one of the questions I’ve been asking myself is whether we should have one estimate per story (or product backlog item) or more than one.

Our development team is new, and – at least in the beginning – not cross functional. The main split is between developers and testers. So I’ve been considering having two estimates: one for development and one for test.

The advantage of doing this is that, I think, it would allow us to arrange the work in Sprints more easily, especially in the longer term. It makes it more visible that certain combinations of user stories just won’t fit in one sprint, even if the total story points are less than whatever our velocity is.

However, the more I think about it, the more it doesn’t seem such a great idea.

Firstly – Scrum teams are supposed to be, if not fully cross-functional, at least semi cross-functional. If we start off with two estimates, how long before we end up needing another separate one for our UI designers, and eventually one for each team member … I’m hoping that having one estimate will help us remember that we should be striving to be more cross-functional and self-managed.

Secondly, even having two estimates isn’t going to give us a perfectly clear idea of what we can and can’t do within one Sprint. At least in the first Sprints there are going to be tasks that can only be done by one person. So ultimately, it’s probably even more confusing.

I’ve found it difficult to find much discussion around this subject. I’m reading Mike Cohn’s book Agile Estimating and Planning, and he advocates using one estimate except in extreme circumstances. I haven’t found much yet to help me address our situation though.

My proposed solution is to have one estimate and use the Sprint Planning meeting to facilitate discussions of what stories we can and can’t do. One technique I learned on the ScrumMaster course was to have the team work through the highest priority stories, or backlog items, one at a time, breaking each into tasks, and stop when we think we can’t do any more.

Taking this one step further, if the team estimate and accept the tasks as we break down the stories, it should become clear when certain team members are “full”. It’ll be interesting to try this and see if it works, particularly how we can rearrange tasks and choose extra stories to fill up the time of the other team members. As with everything else I’m going to try it and see … watch this space for updates.

Can you estimate tasks as zero?

In an excerpt from Agile Estimating and Planning by Mike Cohn, I came across an idea I hadn’t heard about before: using 0 to estimate really small items.

Although we haven’t started estimating the items in our still-in-creation Product Backlog yet, I am intending to try the Planning Poker technique using the Fibonnaci number sequence to estimate the size of each item. In the book, Mike Cohn suggests including 0 in the estimation scale. Sound crazy? It did when I first read it!

But read on … it does have benefits:

If all estimates must be within a certain range, assigning non-zero estimates to tiny features limits the size of the largest features.
If the work to do something really is that insignificant, assigning a 0 to it will avoid impacting the team’s velocity. Assigning a 1 may mean they need to undertake some more significant task to make the same velocity in a later sprint.

Assigning 0 could work as long as everybody on the team recognises that it’s a “free lunch” – and like free lunches, the number of 0 tasks in an iteration must be limited – it’s not an excuse to sneak in lots of extra tasks. 10 x 0 ≠ 0 in reality.

I plan to try this – but if it doesn’t work, I also like the alternative of grouping a number of tiny tasks into a something big enough to represent a 1. Either way, it will be interesting to see how the team accept it.