Reliability is one of the four performance measures that you should consider when building an Agile Balanced Scorecard. The other three are Value, Velocity and Quality.

The big question: Is the Agile Software Development Team delivering what they say they will?

The main Key Performance Indicator for Reliability in an Agile Sotware Development environment is: the difference between Committed Story Points vs. Delivered Story Points over time – you should aim to deliver as close to 100% as possible (no higher, no lower).

If  the team is consistently under delivering then consider the following:

1. Have the team correctly calculated their velocity - if so are they using it to influence sprint commitments??
2. Are the team facing regular sprint disruptions?
3. Are the team bottlenecking at a particular point in the development cycle e.g. QA?
4. Are the team managing expectations?
5. Are the team delivering to the original release plan?
6. Are the team sufficiently empowered to only accept what they feel they can deliver?

Delivery reliability exists at two levels.

1. Ability to deliver the highest value sprint as per their sprint-on-sprint commitments
2. Ability to deliver the project or programme-level commitments i.e. their release plan

Although Scrum ensures that the highest value products are delivered on a ‘sprintly’ basis, we must not lose track of delivery at a project-level. This becomes particularly important when there is a Minimum Marketable Feature(s) (“MMF”) associated with a project i.e. we need to deliver a minimum amount in order to realise a return. Release-level or Programme-level delivery reliability becomes less of an issue when we consider Business As Usual or non deadline-driven/MMF-driven commitments.

If the team consistently over commits and under delivers then you could consider this to be a lead indicator (if it hasn’t happened already) to stakeholder dissatisfaction.

If you are experiencing a lot of disruption during your sprint, you may choose to under commit or introduce stretch tasks into the mix – this will protect the perceived reliability of your team.

If your team bottlenecks at particular points in the development cycle then you should consider introducing Work in Progress limits on different steps e.g. only 3 things are allowed to be ‘In Progress’ or the ‘QA’ column at any given point in time. In order for a developer to start a new task, they might need to help QA a task and see it through to the ‘Done’ column.

I’ve included a template that you can use to track reliability over time using percentage delivery plotted against velocity. Feel free to download it below!

Subscribe to the to be notified when I upload new Articles, Videos, Templates and Tips!

Related Articles:

• Agile Balanced Scorecard – Measuring the Effectiveness of an Agile Team (agile101)
• The Agile Balanced Scorecard – Tracking the Quality of an Agile Team (agile101)
• Using Velocity to measure the Productivity of an Agile Development Team (agile101)
• Minimising the impact of Sprint disruptions (agile101)
• What’s my team velocity? (agile101)