AI Is Quietly Taking Your Memory

The speaker outlines a critical industry pattern where AI companies are simultaneously launching tools designed to capture and retain user context and memory. Claude Tag (launched June 23rd) integrates into Slack as a persistent teammate, while ChatGPT (June 4th) introduced auto-memory that builds profiles of users across conversations without manual curation. Andrew Karpathy called this the third major redesign of AI interaction, after websites and apps—now AI agents are self-contained entities living within organizational tools.

The speaker identifies the underlying incentive: since AI models have become commodities with better versions shipping every few weeks, companies can no longer lock users in through the model itself. Instead, they're racing to own the context layer—the decisions, history, relationships, and operational memory that make each organization unique. This creates a trap that feels like pure convenience on day one (a tool saves 20 minutes, remembers team decisions, learns shorthand) but becomes dangerous over six months when users realize their entire operational memory and decision-making context lives inside a vendor's system. The speaker uses the example of note-taking apps (Roam, Obsidian, Notion, Mem, Tana) where shutdowns or pivots meant losing not just notes but the thinking structure built within them.

The critical distinction is that the AI model is the cheap, rentable part (billions to train, but costs users only token-by-token, constantly improving, impossible to own), while context is the expensive, irreplaceable part (years to build, specific to each organization, impossible to recreate if lost). Most users are inadvertently trading away their expensive context to rent the cheap model—exactly backwards.

The solution is to own the context layer in your own infrastructure and let AI models compete to be the brain running on top. The speaker demonstrates his own system: a local folder of plain text markdown files containing projects, people, decisions, and cross-connected information, with AI agents running as contracts (also plain text) on top. This architecture is inspectable (readable plain text), portable (just files on a disk), and model-neutral (can swap Claude, ChatGPT, Codex, or Gemini without changing anything). He deliberately turns off auto-memory features in both Claude and ChatGPT, controlling what gets saved in his folder based on his own rules rather than the AI's decisions.

The speaker calls his system myPKA (personal knowledge assistant) and offers the entire scaffold freely. He notes he uses the command-line interface rather than the pretty chat app specifically for more control—control over what the agent reads, which model runs, and what can touch the folder. While this seems technical, he argues it's not intimidating for non-coders; it's just typing what you want and having it done.