Agile was a starting point, not the destination

The Agile Manifesto was written in 2001. Extreme Programming was first introduced in 1996. They both set out a set of guiding principles that influence software until now. But in 2022, it is time to move beyond- and away from Agile. Let us set out why!

The Agile Manifesto starts with the words:

"We are uncovering better ways of developing
software by doing it and helping others do it."

Despite this, a generation of practitioners has interpreted the principles in the way a religious fundamentalist reads scriptures: as if their precise interpretation of what the principles mean, are the end destination of software engineering.

On the other extreme, Agile has also been “industrialized”, which is a generous way of saying people figured out they could sell Agile certification, “Agile transformation” and other types of snake-oil that adopt the language, but cement the Status Quo. Change theatre, without any real change.

Agile as a starting point

Agile, as a movement, has its origins in the late 1990ies and early 2000s. Back then, we didn't have the Cloud, we didn't have mature CI/CD systems. The state-of-the-art for version control was CVS. Capture-and-replay testing dominated automated testing, unit testing didn't really become a thing until after 1997, and not mainstream until much, much later. Releasing software was often tedious, time-consuming, and expensive.

Many of the things we take for granted in modern software engineering simply did not exist. XP and Agile are products of their era. They should be viewed as a starting point, not the destination.

Agile is dead, long live Agile

Many Agile Principles are timeless. But many of the practices they imply, or that have since been industrialized, are outdated.

Working to a 2 week or 30 day Sprints, with a demo, or release at the end, as orthodox Scrum prescribes? I'm sorry, your organization is asking to become roadkill in today's fast-moving marketplace. High-performing engineering organizations release to production several times a day.

Do you have someone with the title “Scrum Master”? Do you need to give someone a formal title and a role to ask 3 questions for 15 minutes a day?

Is there a lengthy backlog of “stories” or requirements, rather than business hypotheses, to be tested? Stories and requirements are guesswork. Implementing a long list of guesses without measuring their impact is no better than throwing stuff at a wall to see what sticks. Feedback cycles are the primary driver of improved product delivery. Not having one, or not measuring it, is akin to suicide.

We know what works, and we have the tools

Let's not throw the baby out with the bath water entirely. The Agile Principles have stood the test of time. They are still useful, valuable, as principles. But let's discard some baggage we picked up along the way, and use some knowledge & tools we likewise picked up along the way.

Let's look at a few of the Agile Manifesto principles (abbreviated, some omitted) through a modern lens:

Agile Principles to keep

• Our highest priority is to satisfy the customer: We can do this better than ever, by releasing multiple times a day, experimenting, running A/B-tests, measuring the impact of our daily changes.
• Welcome changing requirements: This is a given. Requirements implicitly change with every new hypothesis proven or disproven, every iteration and every improvement.
• Deliver working software frequently: The manifesto spoke about "every couple of weeks" as a dream aspiration. This is now outdated, but can be updated to "daily".
• Business people and developers must work
  together: sometimes the developers are the business people.
• Build projects around motivated individuals: autonomy, purpose, mastery are the foundations of successful individuals and teams, so yes, still applies.

Agile Principles to discard

• Working software is the primary measure of progress: no. Measurable progress towards our goals is the primary measure of progress. Working software is table-stakes.
• Face-to-face is the most effective means of communication: kind of. But today we have video-conferencing, live pair-programming IDE plugins, Miro Boards and numerous other tools that actually enhance our ability to collaborate beyond what is possible in a shared physical space.

Post-Agile: People, Purpose, Flow

There is much we can take from Agile. But there is also much we can do to move beyond agile. And what I write here is definitely not the last word in ways of working.

We know from Agile that self-organizing teams, with autonomy to control their domain, create empowerment and motivation. We know this reduces impediments. Likewise, we also know teams do not exist in a vacuum. There are often other teams we must work with. Agile provides few answers to this. However, Team Topologies has some good suggestions on how to leverage, rather than become a victim of Conways Law.

For purpose, there is nothing like giving people a clear goal, preferably a goal they can measure their progress against. A combination of collaborative bottom-up and top-down OKRs can provide a solution, but is certainly not the end of the line, and perhaps only yet again a starting point. But if a team has clear Key Results to work towards, where a goal is specified, but the means and method are not, you will have a team that is almost automatically self-managed & goal-directed.

When it comes to Flow towards our goals, our modern toolbox is equipped with endless tactics & tools to fulfil our purpose: We know working in a tight cycle of hypothesis -> experiment -> feedback -> refine -> go to hypothesis has worked for countless companies that have up-ended industries. The Steve Jobs-fallacy of the all-knowing product-genius who knows better than the market and customers is a fallacy because, as the name implies, there is only one Steve Jobs, and you, are not him.

If the hypothesis-experiment-cycle solves our business-iteration side of the equation, modern automated testing, cloud infrastructure and Continuous Deployment-practices solve the technical side of the equation.

Conclusions

If you wanted to summarize a modern version of the Agile-manifesto in three bullet-points, you could do it as such:

• People: Empowered to fully pursue autonomy, purpose & mastery, guided by a wider context aware of, and allowing for "Intelligent Design" of Conway's Law.
• Purpose: Clarity of purpose, with clear measurable milestones co-designed and agreed upon throughout the journey by the team.
• Flow: Adoption of a tight OODA-loop, built upon the foundations of modern Continuous Deployment-practices and the hypothesis-experiment-cycle.

Agile was a starting point. But we should not be fixed in our mind that the early practices or principles are even a desirable destination, given everything we have learned since.

Moving beyond Agile, is simply acknowledging that we are standing on the shoulders of giants. And those giants are the Agile pioneers who dared stake out a new direction. Now it is our turn to continue the journey.

In a future blog-post, we will describe in more detail how we view modern decision-making in software delivery, and how to implement the hypothesis-experiment-cycle that we have alluded to above.