WTF is a harness?`
A harness is code that wraps around an AI agent to make it more effective. The speaker demonstrates how a harness can automatically investigate production issues by analyzing Sentry warnings, identifying root causes, determining impact, and recommending whether to create tickets or apply fixes.
Summary
The speaker explains that a harness is fundamentally a code wrapper around an AI agent designed to enhance its effectiveness and has changed their perspective on how work gets accomplished. They then walk through a concrete example of a harness in action investigating a document operations issue. The AI agent generated an investigation brief that confirmed a Sentry warning existed, determined it impacted 150 users and is still occurring hourly, and clarified that it's a warning rather than a critical error. The harness enabled the AI to identify multiple potential root causes and even pinpoint a blind spot in a specific function. The agent provided exact locations in the product where the issue manifests and offered verification steps by suggesting how to fetch raw Sentry events to validate the identified problems. Finally, the harness-enabled AI made recommendations about whether to create a Linear issue (determining yes, it should be created and assigned) and whether to enter patch mode for immediate fixes (recommending against it).
Key Insights
- A harness is code around an AI agent that makes it more effective and has fundamentally changed how work gets done
- The harness-enabled AI confirmed a Sentry warning affecting 150 users and determined it's still occurring hourly but is a warning rather than an actual error
- The AI identified not only multiple potential root causes but also a blind spot in a specific function within the codebase
- The harness provided exact product surface locations where the issue occurs and offered specific verification steps using raw Sentry events
- The AI made a determination that a Linear issue should be created and assigned but recommended against entering patch mode for an immediate fix
Topics
Transcript
[0:00] A harness is just code around an AI agent that makes it more effective. It's really like changed my mind about how work gets done. Okay, so let's see what it actually generated. Here's the investigation brief on that edit document operations dropped. I have confirmed evidence. So it's saying, "Yes, there was definitely a sentry warning. It's impacted 150 users. It's still happening hourly. It's a warning, so it's not an actual error." And then it found likely root causes. And so it's identified a couple potential root [0:31] causes as well as a blind spot in this particular function. It's told me exactly where in the product surface the issue is and then how I would actually…
Full transcript available for MurmurCast members
Sign Up to AccessMore from How I AI
Codex assumes connectors from your prompt
Codex can intelligently infer when external API connectors (Notion, Slack, Snowflake) are needed based on user intent and context. The system understands complex requests like pulling Notion documents and cross-referencing them with calendar data, and can either prompt users about plugin connectors or automatically implement them.
AI picks my thumbnails now
A content creator demonstrates using OpenAI's Vision API to automatically analyze video frames and select the most flattering ones for YouTube thumbnails. The frame picker app scores frames on expression, clarity, composition, and impact, then combines selected frames with DALL-E 3 to generate polished thumbnail designs.
Codex built my weekly Spotify playlist from Reddit
A developer created an automated website that solves shared Spotify access issues by building personalized weekly playlists from Reddit's music communities. The system scrapes popular music recommendations from Reddit, ranks them by voting, and automatically generates a fresh playlist every Monday morning.
How the OpenAI team uses ChatGPT Sites daily
Kat, a website product leader at OpenAI, demonstrates how the ChatGPT Sites feature with plugin connectors enables diverse use cases ranging from incident management and team collaboration to creative projects like music discovery and 3D game development. The discussion highlights how Sites serves as both a practical business tool and an infrastructure platform for rapid prototyping and creative expression.
Jev beat an LLM at blitz chess
Jev, an AI system, defeats an LLM at blitz chess by using a two-step analysis process that evaluates the top three moves and their subsequent branches in under a second. While LLMs could eventually solve the same problem, they would require significantly more computational resources and time, making Jev's specialized approach more efficient for time-constrained decision-making tasks.