Accessability Links

When is ‘good enough’ actually good enough? – Automation to the rescue: Continuous BDD assurance

Tom Jones
Behaviour Driven Development (BDD) is an effective method of ensuring that a new software release aligns with business requirements and expectations, but it is often let down by time-consuming manual regression testing, which is unable to provide adequate assurance at the start of the acceptance phase.

Introducing: Automation!

When BDD is used alongside automation it provides continual assurance to all members of the software development team that it is moving in the right direction and answering the right questions – namely: Are we producing software that helps solve business problems?

Automation provides timely feedback to sponsors and stakeholders about the development of software (as opposed to the staggered feedback one may receive from manual regression testing) for continuous integration.

As questions such as the one above can be answered in real-time, if any answer other than ‘yes’ is flagged up the team and sponsor can immediately halt the proceedings to get things back on track. An answer of ‘no’ or ‘maybe’ indicates that the business problem in question has not been clearly defined, meaning the solution everyone is working towards is not concrete and requires further clarification.

Apart from ill-defined business solutions, certain team members running behind schedule can also cause there to be confusion regarding if software is meeting requirements. This means teams further down the line are acting reactively, rather than proactively, producing solutions to defects they think need fixing rather than solutions to defects they know do, deviating away from the working solution.

So automation can help to overcome such hurdles as those listed; surely everyone is using it? Well, no…

Let’s get some stats:

According to a EuroSTAR 2014 report, just under half (48%) of software testers use test automation during development all/most of the time. Most use automation at some point in the software development lifecycle (SDLC) – but this ultimately means that around 50% of software is not subjected to continuous test automation during development.

… This means that the majority of software does not undergo continuous test automation, which can make for a lack of certainty in terms of solving the business problem. But on top of this, only 30% start test automation during the requirements or design phases.

This is where BDD is needed most, as it can help with clear communication and requirements gathering. BDD highlights to engineering teams when regression occurs, helping ensure software bugs do not linger for longer than is necessary.

But the defect tracking process is not perfect and could do with undergoing some changes; we think the following should be made:
  • No defect is to be assigned a priority
  • No defect is to be recorded in a separate defect tracking tool
  • All defects are fixed 
Points one and two ensure that defects are all treated equally and flagged-up in unison. But, for point 3 to occur practical guidance is needed by the engineering team on a day-to-day basis (which automation can happily facilitate). In order for this to happen it must be a team choice, not a management one (so BDD helps here, too).

With the goal of a zero known defect release in mind, we have seen success in living by the following three rules:
  • The team will attempt to fix every outstanding known defect by the end of each day 
  • The team will guarantee to fix every outstanding known defect by the end of the working week and always before delivery 
  • The team will never allow the open defect count to go above three at any one time 
Pretty reasonable goals, we think – especially with the use of BDD and automation to help teams along the way. We’re not naïve, though – we realise that some of these changes will have a major impact on all those involved and may require some to completely overhaul their software testing processes. We’ll look at these impacts in another blog, but hope that the above can help to refine further the process of BDD and how it can aid with zero-known defect software releases.

Read our related blog: Impacts of BDD on tools and processes
Add new comment

Meet the team

Back to Top