You don't have to review every AI-generated PR
The speaker describes an AI agent framework that automatically reviews pull requests by scoring them across six risk factors using markdown-based instructions and skills. PRs are classified as low (0-24 points), medium (25-64), or high risk (65+), with only medium and high risk requiring human approval.
Summary
The speaker explains how to build AI review agents using a framework that relies on writing instructions and skills in markdown format. Once an agent reads the code in a pull request, it evaluates risk across six dimensions: the size of the change surface and blast radius, whether changes are easily reversible, whether the PR touches data security and if it addresses security concerns, whether it impacts operations, and the verification gap (whether tests are complete and CI has passed). The agent runs a script to calculate a composite risk score. The scoring system uses three tiers: low risk (0-24 points), medium risk (25-64 points), and high risk (65+ points). This tiered approach allows teams to skip human review for genuinely low-risk changes while ensuring that medium and high-risk PRs still receive human attention, making the review process more efficient.
Key Insights
- Creating AI review agents requires only writing instructions and skills, with markdown being particularly suitable for this purpose
- The agent evaluates risk across six specific dimensions including change surface size, reversibility, data security impact, operational changes, and verification completeness
- The framework uses a numerical scoring system where PRs below 24 points are considered low risk and automatically approved without human review
- Medium and high-risk PRs (25+ points) require human approval, creating a hybrid approach that eliminates unnecessary human review while maintaining safety gates
- Verification gaps are assessed by checking whether tests are complete and CI has passed, factoring into the overall risk calculation
Topics
Transcript
[0:00] Writing these agents is not that hard. It is literally just writing instructions and skills. That is what it takes to create an agent like this and that's why I like this e-framework cuz you can do a lot of it in markdown. And then once it reads the code, it scores the risk. And so it looks at six things. How big is the change surface and blast radius? Is it easily reversible? Does it touch data security and does it cover data security? Does it change anything about our operations? And verification gap is like are our tests complete? Did CI complete? Can we actually test a [0:31] couple ways that this is true? And then it…
Full transcript available for MurmurCast members
Sign Up to AccessMore from How I AI
Enterprise AI fails on governance, not the model
The speaker discusses the importance of skill management and governance in enterprise AI platforms, emphasizing that enabling people to build skills is insufficient without proper monitoring, maintenance, and telemetry. The focus is on providing automatic platform-driven suggestions to keep skills current and using data insights to promote high-value skills while deprecating low-value ones.
Onboard your employees to Cowork in 15 minutes with this system
The transcript describes a workstation operating system that streamlines employee onboarding through an intuitive chat-based interface. The system automates tool connections, role confirmation, colleague mapping, calendar integration, and personalization to get new employees productive from day one.
Your prompt is a spec, and a good spec is the whole game
A prompt acts as a specification that defines quality criteria for AI outputs. By clearly articulating 5-10 defining characteristics of what makes something good (whether a garment, photo, or illustration), you provide the AI with the same level of detail a human creator would need to produce excellent results.
How I use ChatGPT to run my fashion business
Yana Welander demonstrates how she built an AI-native fashion brand using ChatGPT and Codex as her technical co-founder, automating design, production, and business operations from sketches to manufacturing and e-commerce. She shares how AI unlocks previously impossible garments while highlighting remaining challenges in pattern generation and realistic image rendering.
The biggest barrier to AI adoption is not fear of technology. It's the absence of muscle memory.
The speaker argues that the primary barrier to AI adoption is not fear but lack of muscle memory—the habitual use of AI tools. Success requires building collaborative practices through empathy, accessible training, and establishing regular workflows rather than overcoming technological anxiety.