How Intercom 2X'd engineering velocity with Claude Code | Brian Scanlan
Brian Scanlan from Intercom explains how they doubled their R&D team's pull request throughput in 9 months by implementing Claude Code across their engineering organization. He demonstrates their comprehensive approach including skills development, telemetry tracking, and quality controls that enabled 2x velocity gains while maintaining code quality.
Summary
Brian Scanlan, a senior principal engineer at Intercom, discusses how his company achieved a 2x increase in engineering throughput by fully embracing AI-assisted coding with Claude Code. Intercom made this transformation after recognizing AI as an existential opportunity following ChatGPT's release, with leadership setting ambitious goals for doubling R&D output measured by pull requests per R&D employee.
The implementation involved creating a comprehensive skills repository with hundreds of custom tools, from basic PR creation to complex tasks like fixing flaky tests. They built extensive telemetry using Honeycomb and session analysis to track skill usage and developer progress. Key innovations include automated PR description quality control, CI/CD integration, and treating their internal AI adoption like a product with proper instrumentation.
Scanlan demonstrates their approach by live-coding a simple redirect feature, showing how their system enforces quality standards through hooks and skills. He explains their philosophy of 'agent-first' work and removing barriers to AI adoption while maintaining high engineering standards. The session includes discussion of their flaky test fixing skill, which evolved from basic automation to a sophisticated tool that can handle complex debugging scenarios.
The conversation extends to customer-facing implications, with Scanlan showing how they're building CLI tools and agent-friendly interfaces for their products. He emphasizes the cultural transformation, describing increased excitement and productivity across teams, while acknowledging significant token costs that they treat as worthwhile investment. The approach has enabled engineers to tackle previously intractable technical debt and achieve 'backlog zero' on long-standing issues.
Key Insights
- Intercom achieved a 2x increase in pull requests per R&D employee in 9 months after implementing Claude Code organization-wide
- Scanlan argues that imagination, not tools, became the primary barrier to productivity after AI models like Opus reached sufficient capability
- The company uses LLM judges to evaluate pull request description quality and found that AI-generated descriptions were initially terrible, requiring custom skills to maintain standards
- Intercom distributes AI skills through internal IT systems rather than Claude Code's plugin mechanism because they found the official system too flaky for enterprise deployment
- Scanlan claims their flaky test fixing skill evolved from basic automation to distinguished engineer-level capability by incorporating self-learning and progressive discovery
- The team treats AI token costs like hiring whole new offices of people but considers it worthwhile investment during the current alpha phase
- Scanlan reports that Stanford research group analysis of their code showed quality was actually improving with increased AI usage, contrary to common concerns
- He argues that 'backlog zero' is now realistic for engineering teams because the cost and time barriers to tackling technical debt have been dramatically reduced
- Scanlan describes re-implementing a Go microservice in Ruby as a single Claude Code session, work that previously would have required roadmap planning and team coordination
- The company believes all technical work will become 'agent first' and is setting deadlines for this transition rather than gradual adoption
- Scanlan argues that SAS products need to become more agent-friendly with better CLIs, APIs, and self-service capabilities to remain competitive
- He reports having 'the most amount of fun in my career over the last 3 months' due to the ability to quickly realize previously impossible ideas
Topics
Full transcript available for MurmurCast members
Sign Up to Access