Switching from Waterfall to Agile
Many Companies Continue to Struggle with Modernizing their Engineering Practices
Welcome Leaders!
It might seem like the question of waterfall or agile is settled. Old news. Everyone picked a side and moved on.
I have an update for you: it’s not. At least not for everyone.
The question actually comes up surprisingly often when I talk with CTOs.
So, as usual, I’ve got a bit of advice for those who are wrestling with this question.
Cheers,
Decisions, Decisions
Believe it or not, many companies still struggle in 2024 with moving away from the old Waterfall-style approach to software development into more modern Agile practices.
Often, these companies look at Agile as a potential option in the hopes that it will cure some of their ills, but at the same time they cling (often quite desperately) to the legacy Waterfall approach they know and so dearly love.
Why?
Well, it’s crucial to remember that for these companies Waterfall works! They’ve been running the model for years (whether they knew what it was at the start or not), they have customers, the company is alive and more or less thriving.
So why change everything?
But they also know that they have some problems. And while perhaps they can’t pinpoint the reasons for some of these problems, they hear inklings from the rest of the world that this “Agile thing” might actually be worth something.
Usually these companies end up in a kind of gray-zone / purgatory. They are still Waterfall at their core, but they keep making small attempts at being Agile. The unfortunate part is that being in the gray-zone is actually worse than being fully in one camp or another.
Why do companies end up in this state and what can they do to fully move into Agile?
Let’s look at some of the key drivers and then walk through some solutions.
Key Drivers for Companies Staying Waterfall
It’s always worked for them
Their customers won’t let them be Agile; customers might actually like yearly releases, for example
There are a lot of waterfall people in the company
They lack the expertise at making a plan to switch to Agile
They believe the negative stories about Agile
They feel like they are “too busy” to make the switch
They can’t articulate the benefits of Agile to themselves
They tried Agile once and it “failed”
I could go on, but I’ll stop there.
Let’s assume some (or even all) of these factors apply to a company and they just aren’t making the switch to Agile. What do you do as a CTO or Leader of this kind of company?
What follows are 14 “hard truth” answers I’ve discovered from years of watching these scenarios play out.
14 Hard Truths About Switching to Agile
You have to decide whether Agile is worth it for the company & the industry they play in. Not every company needs Agile. It’s hard to admit that Waterfall is OK for some businesses, but it is the truth.
If you decide Agile is needed, then the next thing to decide on is the timeline you’re willing to accept to make the transition. Is this a 3 month thing you’re trying to achieve? Because that’s unlikely. But are you OK with waiting 3 years for it? That might be too long.
If you want to move with any kind of speed you have to exit or move out of the critical path most if not all of the Waterfall people. It sounds harsh, but you’re probably not going to convert many of them. And if you’re moving fast you don’t have time to waste trying to convince them that Agile is worth it.
Think carefully about the upside of Agile to the business, its customers, the product team, engineering team and other stakeholders. Write this down and socialize it with everyone. This is your north-star and your “end state.” Always keep it front and center.
Similarly, write a list of everything that’s broken about the current Waterfall model. Socialize this You’ll need to remind yourself and your teams why you’re switching to Agile or people will just forget and assume the “good old days” of Waterfall were so much better.
Build an aggressive plan to switch. I say aggressive because you don’t want to half-ass this thing. Either fully do it or just stay Waterfall. The longer you take to make the switch the more likely it is that you will never do it fully.
Don’t try to reinvent the wheel with Agile. Just pull something “off the shelf” like SAFE (but a simpler version of it) and go with that. Don’t invent new terms, your own frameworks, or try to use this as an opportunity to be an Agile guru. Others have done it better, I assure you.
And don’t waste a ton of time on teaching core principles of Agile. Actually practicing Agile is the best way to learn it. Set up your first daily standup ASAP, run it, get feedback and iterate. You can offer people books on Agile if they want to learn more in their own time.
Let the team and other stakeholders know that things are going to be broken for a while as you make the change to Agile. There’s no way to make the switch perfectly smoothly. There will always be some trouble spots with people, process, deliveries, roadmap and so forth.
No leader trying to move from Waterfall to Agile is popular all of the time. Know that you’re going to make some people upset and they will complain. That just comes with the territory. This is definitely a “you have to break some eggs to make an omelet” scenario.
As you go through the process you’re going to need checkpoints to ensure you are making progress. There isn’t one answer to what the right milestones are that works for every company. You have to decide what success looks like at each milestone for your team, company and customers.
Agile consultants can help for sure. But only if they are well-matched to the team and business. Don’t hire any aspirational & lofty consultants unless you have several years to make the switch. Hire practical experts who are diplomatic but believe in your timelines.
As the CTO or Technology Leader you have to have 3 key soft-skills to lead the change: communication/diplomacy, patience, and the temperament/ability to make hard decisions like exiting staff who aren’t bought in.
This is the harshest truth of all: even if you succeed in shifting to Agile you are unlikely to get many accolades for pivoting the team/company. It will be several years later where the biggest benefits show up and by that time people will forget all about how hard you worked. So don’t do it for the wrong reasons.
Like I said, harsh truths… 😃
See, the misunderstanding that many CTOs and Agile Consultants have about legacy Waterfall companies is that they think these organizations will switch to Agile when the pain of Waterfall gets big enough.
The problem is that it’s not the pain of Waterfall that eventually drives these companies to switch. The pain is actually rarely big enough in Waterfall-mode. And, it’s also not the benefits of Agile which drives a company to switch, since those are very well known and typically do NOT compel legacy businesses to change.
You would think it would be either one or the other: pain or benefits.
But, no, its none of those.
Closing Thoughts
Most of the time what drives companies to change is one person with a vision, who has seen both sides of Waterfall & Agile, and knows the upside of switching to Agile is huge down in their bones.
It’s a transition that requires a hero. Logic doesn’t work as I noted above. It takes a big effort from 1 or 2 “hero” people to drive an Agile transformation forward and realize the benefits for the business.
And that’s why so many companies fail. Because these heroes are not easy to find.
If you’re a CEO and want to transition to Agile look for the hero.
If you’re a CTO and need to transition to Agile then you have to become the hero.