You Bought the AI Tools. Nothing Changed.
Where engineering leaders are going wrong with AI adoption…
I was at an AI conference the other day talking to a nice guy with a ton of energy and an enormous amount of enthusiasm about AI. He was fired up about the AI adoption work his engineering team had done over the last 18 months.
A total revamp of the SDLC, he told me. He used the phrase “AI-native” more than once.
It was a familiar conversation. In fact, I have some version of this discussion with CTOs all the time nowadays.
Technology leaders everywhere have built sleek versions of AI-native engineering teams and for once, they have the budget to chase this vision. The AI hype-cycle has given technology leaders unusual permission to spend on new tools, new workflows, new copilots, new agents, new internal platforms.
And in many cases, CTOs are pulling it off. They are modernizing the SDLC. They are getting engineers to use AI. They are changing how code gets written, tested, and deployed.
But that’s also where the trap begins. 🪤
Difficulty Driving Results with AI
Later at the conference, the same guy told me about an argument he’d recently had with his CEO.
She was supportive of his AI adoption work. She understood the long-term value. And she believed the engineering organization needed to modernize itself. But she had a more immediate problem: customer churn was getting worse.
Users were complaining that the product had gotten stagnant. Competitors were moving faster. And the company had not released a truly compelling new feature in several quarters, if not an entire year.
She was ticked off because Mr. Conference Guy (who was the CPTO) had rebuilt parts of the engineering team, but the business was still waiting for what it actually cared about: new features & new products.
He knew she was right, but I could tell by the tone of his voice that he couldn’t admit to himself that his high-end, “AI-native” engineering team wasn’t driving actual business value.
The Key Mistake Tech Leaders Are Making
The mistake I’m seeing so often from technology leaders like Mr. Conference Guy is making AI adoption about engineering process & tools instead of new projects & features.
Tech leaders are redesigning the SDLC, rolling out copilots, building AI guidelines, creating prompt libraries and running enablement sessions. They’re asking engineering teams to experiment with agents, code generation, automated testing, documentation, ticket writing, code review, and every other shiny improvement they can think of.
None of this is bad. Some of it is probably necessary. But it is not the same thing as moving the business forward.
Most of the (above) activies are about the engineering department itself, not the business. This work makes the machinery of dev look more modernized. It gives the CTO something to point to. It creates the feeling of momentum. But the business does not really care whether the SDLC is AI-native or not. The business cares whether the most important product innovations are happening.
In other words, a lot of engineering organizations are becoming more active without becoming more effective. There is more tooling, more experimentation, more internal change, more AI language in meetings, more demos, and more evidence that engineers are “using AI.” But the roadmap still looks the same.
CEOs (like the conference guys boss) are sitting around waiting to see impact and don’t realize they aren’t going to get it as fast as they wanted.
That is the danger of making AI adoption a process & tooling issue first. You can spend months improving how engineering works without materially changing what engineering delivers.
It is a little like reading every workout plan, buying a garage full of gym equipment, tracking your protein intake, and then wondering why you are not in better shape. At some point, the question is whether you are actually working out & eating right.
Focusing on Projects is a Better Approach
By the end of the conference the CPTO had started to shift his mindset a little bit. Here’s what I said to him:
It’s great if AI has improved your PRs and sped up cycle time. Those are excellent and necessary metrics to track if you want to manage engineering better. But those are just for YOU.
The real question is whether AI is helping deliver projects that actually matter. Not random internal experiments, but the important projects. The ones tied to revenue, retention, margin, customer experience, product velocity, or competitive advantage.
In that sense, AI has not changed the fundamental job of technology leadership. You need to figure out the priorities, make projections of ROI & impact, and decide on complex trade-offs.
AI can accelerate the work and make teams faster. But it doesn’t remove the need for executive judgement. You still need to decide which initiatives deserve disproportionate attention because they can actually move the company forward.
Technology leaders should focus on delivering high-value projects first, and allow the process & tooling to catch up instead of focusing on building the ultimate AI-native engineering team.
Where AI Actually Creates Leverage
Some projects are much better suited for AI than others.
Not because AI magically makes them easy, but because they contain the kind of work AI is good at: analysis, translation, summarization, documentation, refactoring, testing, dependency mapping, workflow cleanup, etc.
Let’s look at some real-world examples every CTO can apply AI to:
New Products & Major Features — This is where the conference guy probably should’ve been focused. AI is especially useful when you are building from zero: generating prototypes and accelerating the first version of something customers can actually use.
Modernization — AI is getting much better at reading code, explaining legacy logic, identifying risky areas, suggesting refactors, and helping teams break down large modernization efforts into smaller, safer chunks.
Internal Reporting — AI is especially good at this. And for sure CTOs always have a backlog of reports to produce for internal stakeholders. Use AI to get the analytics & reporting factory floor moving again.
Quality — AI can get you out of a quality hole. From analyzing your bug backlog to actually fixing bugs, use AI to show your CEO and head of customer support an impressive reduction of outstanding bugs.
Integration Work — Integrations are full of tedious work: understanding APIs, mapping fields, translating business rules, handling edge cases, writing glue code, testing failures, and documenting how systems actually talk to each other. AI can help speed this up.
Switching from Process to Projects
Here’s the recommendation I gave to the conference guy:
Make a list of your key projects, whether they are already on the roadmap or sitting somewhere in the backlog, and score them across five factors.
Factor 1: AI Leverage
How much can AI actually help with the work? Code generation, testing, refactoring, documentation, analysis, API mapping, requirements cleanup, etc.
Factor 2: Business Impact
Does this project matter? Will it improve revenue, retention, margin, customer experience, product velocity, or competitive positioning?
Factor 3: Measurable Outcomes
Can you prove AI helped? Faster delivery, lower cost, fewer defects, more test coverage, less manual work, or more value shipped.
Factor 4: Available Context for AI
Does AI have enough to work with? Code, tickets, specs, architecture notes, product docs, customer feedback, API docs, test results, and so on.
Factor 5: Execution Risk
Is this project likely to be slowed down by messy requirements, legacy code, poor documentation, manual QA, unclear dependencies, integration work, or heavy coordination?
The uncomfortable question for CTOs is: are you using AI to make engineering look more modern, or are you using it to move forward the projects that actually matter? Because those are not the same thing.
Better process is useful. Better tooling is useful. But the real test is whether AI changes the trajectory of the org. If your most important projects are still moving at the same speed from the perspective of your customers, then you probably have an AI “activity plan” not a real AI transformation plan.
Remember you’re transforming the success of the business, NOT how engineering operates — that comes later.