TL;DR: Agile Metrics Summary Paper

TL;DR: Agile Metrics
Suitable agile metrics reflect either a team’s progress in becoming agile or your organization’s progress in becoming a learning organization.
At the team level, qualitative agile metrics typically work better than quantitative metrics. At the organizational level, this is reversed: quantitative agile metrics provide better insights than qualitative ones.
Good Agile Metrics
Generally speaking, metrics are used to understand the current situation better as well as to gain insight on change over time. Without metrics, assessing any effort or development will be open to gut feeling and bias based interpretation.
A metric should, therefore, be a leading indicator for a pattern change, providing an opportunity to analyze the cause in time.The following three general rules for agile metrics have proven to be useful:
The first rule of tracking meaningful metrics is only to track those that apply to the team. Ignore those that measure the individual.
The second rule of tracking metrics is not to measure parameters just because they are easy to follow. This practices often is a consequence of using various agile tools that offer out-of-the-box reports.
The third rule of tracking metrics is to record context as well. Data without context, for example, the number of the available team member, or the intensity of incidents during a sprint, maybe turn out to be nothing more than noise.
For example, if the (average) sentiment on the technical debt metric (see below) is slowly but steadily decreasing, it may indicate that the team:
May have started sacrificing code quality to meet deadlines or
May have deliberately built some temporary solution to speed up experimentation.
While the latter probably is a good thing, the first interpretation is worrying. (You would need to analyze this with the team during a retrospective.)
Good Qualitative Agile Metrics: Self-Assessment Tests
If you like to track a team’s progress in adopting agile techniques and processes, self-assessment tests are well-suited for that purpose. For example, I like to use the Scrum Checklist by Henrik Kniberg.
All you have to do, is to run the questionnaire every four to six weeks during a retrospective, record the results, and aggregate them:
Age of Product: Age Metrics: The good, the bad, and the ugly – Scrum selfassessment
In this example, we were using a kind of estimation poker to answer each question with one of the three values green, orange, and red. The colors were coded as follows:
Green: It worked for the team.
Orange: It worked for the team but there was room for improvement.
Red: It either didn’t apply, for example the team wasn’t using burn down charts, or the practice was still failing.
If the resulting Scrum practices map is getting greener over time, the team is on the right track. Otherwise, you have to dig deeper to understand the reasons why there is no continuous improvement, and adapt accordingly.
In addition to this exercise, I also like to run an anonymous poll at the end of every 2-week-sprint. The poll is comprising of three questions that are each answered on a scale from 1 to 10:
What value did the team deliver last sprint? (1: we didn’t deliver any value, 10: we delivered the maximum value possible.)Agile metrics: customer value delivered – Age of Product
How has the level of technical debt developed during the last sprint? (1: rewrite the application from scratch, 10: there is no technical debt.)
Are you happy working with your teammates? (1: I am looking for a new job, 10: I cannot wait to get back to the office on Monday mornings.)
The poll takes less than 30 seconds of each team member’s time, and the results are, of course, available to everyone. Again, tracking the development of the three qualitative metrics provides insight into trends that otherwise might go unnoticed.
Good Quantitative Agile Metrics: Lead Time and Cycle Time
Ultimately, the purpose of any agile transition is to become a learning organization, thus gaining a competitive advantage over the competition. The following metrics apply to the (software) product delivery process but can be adapted to various other processes accordingly.
In the long run, this will not only require to restructure the organization from functional silos to more or less cross-functional teams, where applicable. It will also require analyzing the system itself, for example, to figure out where queues impede value creation.
To identify the existing queues in the product delivery process, you start recording five dates:
The date when a previously validated idea, for example, a user story for a new feature, becomes a product backlog item.
The date when this product backlog item becomes a sprint backlog item.
The date when development starts on this sprint backlog item.
The date when the sprint backlog item meets the team’s ‘Definition of Done’.
The date when the sprint backlog item is released to customers.