HS134: Dodging the AI Iceberg: Midcourse Corrections
John Attil Johnson and John Burke of Numerides discuss when and how to make mid-course corrections to IT and AI strategies. They cover triggers for strategic reassessment, the value of wargaming and scenario planning, and how business imperatives should drive technology decisions. The episode emphasizes structured, recurring strategy reviews rather than reactive responses to crises.
Summary
In this episode of Heavy Strategy, co-hosts John Attil Johnson and John Burke of Numerides explore the topic of mid-course strategy corrections, with a particular focus on AI and IT strategy. They open by identifying several non-emergency categories that should prompt strategic reassessment: things that were unclear at the outset but are now visible, developments that arrived faster than anticipated, projects that are over budget or under-delivering, and unexpected surprises—both positive and negative.
The hosts use AI deployment in help desks as a concrete example, referencing research showing roughly 20% productivity gains for entry-level help desk work but weak to negative correlation for higher-level staff. They argue this kind of differential outcome should prompt organizations to revisit where and how they are applying AI, rather than assuming uniform benefits across all use cases.
A significant portion of the discussion focuses on the importance of after-action reviews and the 'five whys' methodology—drilling down into root causes rather than simply noting that something didn't work. They recommend at least quarterly strategic check-ins at the leadership level to assess whether strategy is on track and whether external conditions have shifted. They also connect IT strategy back to business imperatives, noting that if the core corporate objective changes—say, from margin improvement to revenue growth—the technology strategy must be re-examined from the ground up.
The hosts advocate strongly for wargaming as a proactive tool, arguing that organizations should red-team their strategies to identify contingencies in advance. They cite the example of clients who had pre-built pandemic response plans and were able to activate them smoothly when COVID-19 hit. They extend this logic to current geopolitical and supply chain risks, such as helium shortages affecting MRI centers, TSMC disruption affecting hardware procurement, and rare earth mineral dependencies on China.
They also discuss how technology advances can render entire business processes obsolete, warning against over-investing in technically enabling processes that new technology will soon eliminate. This is illustrated through the anecdote of Cooper Union's original architects pre-building an elevator shaft in the 1800s, anticipating future need. The hosts close by noting that course corrections should also prompt a review of whether the IT organizational structure—particularly its degree of siloing—is still fit for purpose.
Key Insights
- The hosts argue that research shows AI produces roughly 20% productivity gains for entry-level help desk work but has a weak to negative correlation with productivity at second and third support tiers, suggesting AI benefits are not uniform across organizational levels.
- Johnson contends that many IT projects are launched without a clear definition of what they are supposed to accomplish, making it difficult to assess whether a course correction is needed.
- Burke argues that wargaming IT and business strategies—where someone red-teams the plan with simulated external events—can convert reactive course corrections into the activation of pre-planned contingencies, reducing the cost and chaos of mid-stream pivots.
- The hosts assert that if senior executives are repeatedly asking about a technology or risk that is not currently on the IT strategy roadmap, that is a reliable signal that a strategic reassessment is overdue.
- Johnson argues that organizations should explicitly map critical resource dependencies—such as rare earth minerals, helium, or TSMC silicon—and model how disruptions to those resources would force changes to IT and business strategy.
- Burke contends that when designing new IT systems or infrastructure, teams should proactively identify the conditions under which that investment will become obsolete, so transitions can be planned in advance rather than reverse-engineered under pressure.
- The hosts argue that IT strategy must be traced back to corporate-level business imperatives, and that a change in top-level objectives—such as shifting from margin improvement to revenue growth—should trigger a full re-evaluation of the technology strategy stack.
- Johnson suggests that when making a course correction, organizations should also evaluate whether their IT organizational structure, particularly the degree of siloing versus horizontal integration, is still suited to the new strategic direction.
Topics
Full transcript available for MurmurCast members
Sign Up to Access