Incorporating Usability

In my quest to incorporate usability into our Scrum model in an agile way, I came across this article on Agile Usability.

It advocates drawing the design on paper to hammer out the big concepts, and running a heuristic analysis with several usability experts to uncover any major design flaws. The designer and developer then pair program to create the interface. Once a prototype is created, perform small scale usability testing. The requirements can then be adjusted. It also points out that Usability experts can help break down the items on the product backlog.

For us, the concept of having usability break down the product backlog is something we are already starting to do, and definitely something I want to continue. The idea of having our designer pair program with a developer to create a design is interesting. I wonder how he’d feel without Photoshop?? The rest of the cycle makes pretty good sense, isn’t that far from what we did this Sprint and seems like a good way to play once the team is all in London (until then, if we can’t be perfectly agile, I’m happy to live with it!)

I’m still researching, but this one was worth highlighting.

Agile newbies can still add value

I discovered today that even as a relative-agile-newbie, one of the most useful and rewarding things you can do is to try to help another Agile team with their problems. This is a scary concept: letting me and my limited experience with Agile loose to try and advise somebody else on how they can do things.

I’m spending this week working in our Bellevue office, with a group of my colleagues who are just starting out on their first Sprint. They’ve done the CSM course, and they’re just starting to see how quickly Scrum brings problems to the surface.

It’s actually surprising how much value even a relative-agile-newbie can add, particularly within a group of people who want to learn from each other. It’s a great experience, and if you get the chance to work with another team, try it (just make sure they can’t blame you afterwards). I’ve recognised so many things I’ve done, or still doing, that I want to change. I’ve spent time discussing the problems the other team are having, hearing other people’s views on the world, and it makes me think about what the agile principles are, what the Scrum rules are, and how I’m not following them as well as I thought. Things can slip out of your mind and trying to find the tools to solve other people’s problems helps reinforce them back.

The best bit ever is that I think I’ve actually come further than I realised. On the odd occasion, I’ve actually had an intelligent answer. Most of them may have been parroting what somebody else has told me, but this stuff starts to feel like it’s sticking. I’m still a Agile newbie, but I’m becoming an Agile junkie. I’m having fun with this.


Brainstorming “Done”

Being a new Scrum team, one of the things we’ve had difficulty with is actually completing the stories we set out to do in our Sprints. A lot of our tasks were pretty big, they didn’t move very quickly across the taskboard, we were  forgetting things like documentation, and the team wasn’t owning the estimates.

We decided to try something suggested to us by another Scrum team that we’d been talking to, who had been messing with Scrum for a year or so (they work for a different company nearby, we’ve recently started up a conversation and it’s great to get input from outside).

The idea was to brainstorm what it means to be “done”. We started with around ten or twelve criteria like coded, tested, and documentation completed. But when we brainstormed and broke it down further, we ended up with nearly thirty things identified. They’re pretty generic – I’ll list them at the end of this post.

The next step was to go through the list for every story that we took on in Sprint planning. Not every single thing on the list applied to every story, although most of them apply in most cases. For each item on the list, we created a task. This meant that task creation was no longer something that was owned by any one team member, and almost all of the tasks were a lot smaller – for example, we had to call out separately a task to review code, to review documentation, and to review acceptance tests, which we hadn’t done before. In this sprint, we’re tracking progress at a much more granular level, which means it’s far easier to see where the real hold ups happen, and harder to forget things.

This exercise isn’t completed yet though. The next step is to review the critieria at the end of this Sprint, identify the gaps, and fill them in. For example, I don’t think that “documentation written” is specific enough. If our documentation contains screenshots, a flow chart and a technical overview, let’s call that out. It sounds like over-planning, but it’s not – it really helps the team to see clearly what the tasks are and start to share them out more – which I think is one of the biggest challenges of being Agile.

The other thing we need to do is to write a sentence for each criteria to really clarify what it means to have met it. Watch this space for further updates on how this goes!


  • Story written
  • AB tests identified
  • UI designed
  • HTML written
  • Code written
  • Code reviewed
  • Unit tests written
  • Unit tests passed
  • Documentation written
  • Documentation reviewed
  • Acceptance criteria defined with stakeholders
  • Acceptance criteria reviewed and approved
  • Acceptance tests written
  • Acceptance tests published
  • Functional tests written
  • Manual tests carried out
  • Tested accessibility
  • Acceptance test done
  • Automation testing scope defined
  • Automated tests written
  • Automated tests run
  • Bug fixing
  • Regression done after bugfixing
  • Regression testing defined
  • Pseudo localised
  • Pseudo loc testing completed
  • Deployed to lab
  • Ready to present to stakeholders
  • Product development roadmaps, integrating usability and keeping testers busy …

    Halfway through our second sprint, my impediment backlog has 22 impediments in it. At least I have a purpose in life.

    The biggest impediment is not having a server to build and test our code on. This is a pretty major one, which we are frequently bitching complaining about, and doing our best to convince somebody to resolve, with limited success so far. But I remain hopeful …

    The next three are in relation to how the scrum team and product owners work together (yes, that’s right, product ownerS – it’s not ideal, but our Scrum team has to complete work not only on the web UI project that is our main focus but also on some internal applications that can’t be ignored).

    Impediment 1: We have separate workstreams and until Sprint Planning, no idea how they fit together. Our product owners don’t really know how to combine the priorities, and can’t predict beyond the current sprint when they might get any of the things they’ve asked for.

    My proposed solution: we need a development roadmap that lays out the bigger themes that we will work on. Being only two sprints in, we don’t have a predictable velocity yet but we can start predicting date ranges within a couple more sprints. I don’t plan on having something set in stone, but I haven’t read anything that says having a longer term vision is a bad idea – on the contrary, Mike Cohn’s book (Agile Estimating and Planning) promotes having a plan that represents our best view of the future at any point in time. At least, that’s how I read it.

    Impediment 2: we have a visual designer and usability engineer who are not co-located. The usability engineer is based in Seattle. Time difference = -8 hours

    My proposed solution: until they are co-located, they work a sprint or two ahead of us so that our sprints are not held up by slow communication loops. We need to figure out just how this works.
    I’m not sure I’m 100% comfortable with this as a longer term solution. According to Scrum by the book, the team are supposed to figure out everything within the Sprint. In this current Sprint, we brought our designer and usability guru to London and they locked themselves in a room for two days to figure out the designs for the interaction and the interface. They did a good job, and I’m not sure how we could organise this any better. The developers couldn’t start their work until this was done, which seems a bit like mini-waterfall, but I’m not sure if I’m taking this to the extreme. Time will tell.

    Impediment 3: Testers had nothing to do at the start of the Sprint.

    My proposed solution: moving the UI design ahead of the sprint will help with this. Also, we did this a lot better than in Sprint 1 – our testers worked to define acceptance tests early on in the Sprint. I have to admit … I don’t really have any other ideas on how to fix this, and again I’m hoping time will tell …

    Overall, so far this second Sprint seems to be going very well and I’m hopeful we’ll complete what we committed to – the burndown certainly looks really positive!

    (an even bigger acheivement … I’ve actually written a weekly post for two weeks running!)

    Hop, walk, jog, stumble … enough jokes already

    Tomorrow, we’re due to finish our first Sprint. Or whatever you decide to call it. I’ve had many suggestions: stroll, wander, hop, walk …

    On the one hand, I think we’ve done pretty well to not only survive it almost unscathed, but to also actually complete a few user stories. We had a new, still-only-partially-recruited team working on a new project using a new methodology, and in the middle of it we had a major new release of our core website technology which resulted in a lot of unavoidable disruptions. To cap it off, I can’t remember another two week period when more people have spent more time out of the office. I’m not sure if it’s just because Scrum gives you so much more visibility into that stuff but I’ve never known so many training courses to happen in the same space of time.

    On the other hand, I can’t help being a bit disappointed that we weren’t perfect. Even though I was expecting it to turn out exactly this way I was still hoping that we’d do everything we said we would and be ready to demonstrate it at the Sprint review, with everybody there.

    My next challenge is how to turn this into a positive learning experience and not get too weighed down by the negatives. I’m also planning to go back and re-read a lot of the Scrum literature with the benefit of hindsight and some context, and to remind myself of the things I forgot …

    On Sprinting at last!

    We are just into the second week of our first Sprint. I’d been waiting for this for a while, and so it’s been fantastic to no longer just read about Scrum and Agile, but actually start doing it.

    In terms of completing what we said we were going to, I’m pretty sure we are going to “fail” our first Sprint. In terms of what we’ve learned about Scrum and about ourselves, I think it’s been an outstanding success. There’s been two big things I’ve really noticed over the last week and a half:

    Everything really has become very visible – the little problems you didn’t realise you had come up even more quickly than you imagined they would.
    The development team are working together more than I’ve seen happen … well, ever. Not because anybody told them to. They just did it.
    I’ve talked to other people using Scrum and I think that overall we were pretty lucky. I had a lot of time to immerse myself in learning about Scrum before we started, we have a fantastic team that have really bought into the methodology, great support from management, and a whole new project to try out the new techniques. The biggest impediment we’ve got right now is that the team is not yet complete: two new members will start over the next two weeks and the last one will move from Paris in August.

    I fell into a lot of traps this Sprint Here are some of the things that have already come up. Many of them are completely obvious … Duh.

    • To start with, I forgot that we should update task “hours remaining” estimates in Daily Scrum. So, our initial burndown was pretty rubbish. This was easily fixed though.
    • I also forgot that Daily Scrum isn’t just about reporting what Sprint tasks the team have worked on. Saying “I haven’t done anything on the Sprint tasks in the last 24 hours” isn’t enough. What were you doing then? I was wondering why I wasn’t identifying many impediments. This is particularly difficult since two of our four team members are not 100% dedicated, sometimes it’s difficult to determine what is an impediment and what isn’t.
    • I don’t feel like we scheduled for time OOF (holidays and training), conflicts with other meetings, and other big events very well. For example, there was a major release of the website across all of our points of sale in the middle of our Sprint. We didn’t think enough about what preparation work we’d need to do for that, and it bit us on the a** pretty hard. Ouch.
    • An interesting thing that came up today was that our testers wanted to extend the scope of one of the test stories (building part of an automation framework) even though there was a lot of outstanding work on other stories. Being a new team, we aren’t very cross functional – not at all really – so the question is what else are the testers going to do? I jumped at the chance to wear my mean ScrumMaster hat and asked them to figure out how to work on one of the other stories, which had some test work to do on it. As a result, we figured out that there was an impediment there because they didn’t have a server available to test on (urgh – another major impediment right now is that we have some major hardware availability issues. We have been promised some this week, so we’ll see). I think we have to work on this understanding that we commit to the stories and then try to finish them.
    • I feel as if I’ve tried to lead the team too much in this Sprint – in Daily Scrums, I feel as though I’m the one asking what have you done on X? How much is left on Y? Why can’t you do Z now? Perhaps that’s OK since it was the first one and I’m aware that over time they need to ask each other that. I’m hoping that over time the team will self manage more. On a positive note, they have been waiting at the whiteboard for Daily Scrums, which was great. We only had one instance of people being late and that was due to a conflict.
      Although our story estimates were owned by the group, task estimates were done individually. And some of them were pretty atrocious. I want to try and encourage more of a group effort next time.
    • I think to some degree we’re still doing “design-dev-test” – or mini waterfall in Sprints. There’s a big division of work, and as a result we’ve still got five stories in progress and none complete. Over time, I want to try and change this so that we work on stories together. In the first Sprint though, I don’t see this as a big problem.
    • Some of our technical stories were pretty vague. Like “fix bugs” (we had some old bugs) or “I want to automate testing using a framework”. This means we had lots of wiggle room but we don’t really know for sure when to stop and say the story was complete. We’re learning though and we’ve added new stories and set up ways to manage and triage bugs.
    • We need to be more prepared before Sprint planning meeting. We spent quite a bit of time defining new technical stories and estimating stories with missing estimates in Sprint planning. That said, our planning meeting didn’t run over time, so it didn’t cause a huge problem.

    As well as the teamwork, there have been some real positives and things we’ve done that have worked really well:

    • Having a taskboard with post-its and holding Daily Scrums around it worked great. The meetings are short and snappy and having a daily update is fantastic for visibility – I learn more in those fifteen minutes than I could from an hour reading a spec.
    • Creating a product development roadmap has really helped to prioritize stories and create an end goal.
    • Estimating in Story Points has been working pretty well. Planning poker is actually fun, and the team adapted really quickly to questioning each other’s estimates. I think with the benefit of one Sprint, triangulation will get even easier.
    • Just talking about what we are developing instead of completing set documentation is really beneficial – everybody has a far better understanding of how things work. We don’t have prescribed spec reviews, we talk face to face on an ad-hoc basis.

    I could go on for hours that covers the main things so far!

    How many estimates?

    This week, one of the questions I’ve been asking myself is whether we should have one estimate per story (or product backlog item) or more than one.

    Our development team is new, and – at least in the beginning – not cross functional. The main split is between developers and testers. So I’ve been considering having two estimates: one for development and one for test.

    The advantage of doing this is that, I think, it would allow us to arrange the work in Sprints more easily, especially in the longer term. It makes it more visible that certain combinations of user stories just won’t fit in one sprint, even if the total story points are less than whatever our velocity is.

    However, the more I think about it, the more it doesn’t seem such a great idea.

    Firstly – Scrum teams are supposed to be, if not fully cross-functional, at least semi cross-functional. If we start off with two estimates, how long before we end up needing another separate one for our UI designers, and eventually one for each team member … I’m hoping that having one estimate will help us remember that we should be striving to be more cross-functional and self-managed.

    Secondly, even having two estimates isn’t going to give us a perfectly clear idea of what we can and can’t do within one Sprint. At least in the first Sprints there are going to be tasks that can only be done by one person. So ultimately, it’s probably even more confusing.

    I’ve found it difficult to find much discussion around this subject. I’m reading Mike Cohn’s book Agile Estimating and Planning, and he advocates using one estimate except in extreme circumstances. I haven’t found much yet to help me address our situation though.

    My proposed solution is to have one estimate and use the Sprint Planning meeting to facilitate discussions of what stories we can and can’t do. One technique I learned on the ScrumMaster course was to have the team work through the highest priority stories, or backlog items, one at a time, breaking each into tasks, and stop when we think we can’t do any more.

    Taking this one step further, if the team estimate and accept the tasks as we break down the stories, it should become clear when certain team members are “full”. It’ll be interesting to try this and see if it works, particularly how we can rearrange tasks and choose extra stories to fill up the time of the other team members. As with everything else I’m going to try it and see … watch this space for updates.