Non-linear Software Development Workflow

Software development, like many manufacturing processes, is all about work dependancies. We do one thing so that another person can do their job. For example, testers need working software to start their testing. Well, perhaps they need working software to finish their testing.

Here's the interesting thing. In our team (a scrum team working developing a payment switch) the testers don't need working software to finish their testing. We have managed to remove the dependancy between software development and software testing. Let me explain how.

First, however, waterfall. In 1970 a dude (almost everyone working in CompSci in 1970 was a dude) proposed the waterfall model. In this model, we follow a strict series of steps to produce software. The steps are:

(Ignore for the moment that Royce also said that the waterfall model doesn't work for large software projects - almost everyone else ignored him so we can as well :)

In the waterfall model, we have a strict "finish -> start" dependancy. For example, requirements must finish before design must start. This problem is also present in the iterative methodologies (I also dropped "maintenance" from the flow as maintenance is just another loop around the cycle):

However, then agile came along and some very clever people (I've heard this idea from both Alastair Cockburn and Mick Cohn) realised that the dependancy between each stage is not a "finish-start" dependancy but it is a "finish->finish" dependancy. What that means is that testing can start before development finishes, but testing can't finish before development finishes.

This model is really useful in methodologies that have short time-boxed sprints - like Scrum (which, potentially, is not really a methodology but a "Reflective Improvement Framework" but we'll leave Alastair Cockburn with that interesting definition). It's useful because it means that team members can work mostly in parallel during a sprint.

Now, here is the interesting thing. With methodologies that have short time-boxed sprints your team starts getting really good at breaking down features into tiny stories - little pieces of functionality that deliver value to the user or customer. They represent externally visible changes in system behaviour that can be developed and tested (they also might be tokens for work, but that's a discussion for another time). The most important thing is that they are small. Very small. They can be as small as "text field for name displayed on screen" and then the project can have separate stories for data entry, field validation, security, etc. Some people have been known to slice keyboard and mouse navigation for an interface into different stories.

With stories this small it is entirely feasible for the entire team to sit down and very quickly get a testable shared understanding of the story. Then each discipline can go away and start work. The testers can write automated tests for the story and check them into the system. The developers can write code. The BA can resolve any ambiguity and communicate it to both the testers and developers.

This creates what I call a "shared start dependancy with a deadline". The shared start dependancy is the creation of a testable collective understanding in the team. The deadline is the end of the iteration.

Easy eh! Well no. In our team - where this happens regularly - there were several things we needed before this behaviour emerged. The things were:

• An automated test suite where testers can specify tests without reference to the user interface. We used Concordion.
• An automated test suite where testers can check in tests that aren't run by default (to give the developers a chance to build the feature before the continuous build system starts failing the tests).
• We used the Jenkins CI server. Tests were stored in a Git repository - the testers used SourceTree to check their tests in. 
• In the Maven build file we told Maven to only run Concordion tests called "indexTest.java" and then referenced active tests from the those files using the "c:run" annotation.
• One Concordion hint: use Map<String, Object> as your return value for most of your fixtures. Look at returning a map result - but sadly no direct link to the correct section.
• Maturity in slicing stories smaller and smaller. That took the current team 6 months of development - but we had a very low level of scrum experience when we started.
• Much collaboration and a realisation that it was possible for a tester to specify the tests before the development started.

Wow. Thanks for reading this far!