Claude Reads My Handwriting and Sketches Now
The creator demonstrates a workflow for capturing handwritten notes and sketches using PDF Expert on iPad, syncing via iCloud, and having Claude AI read and act on the annotated PDF files. The system connects to a local 'myPKM' folder where an AI orchestrator named Larry processes inputs and outputs. The workflow eliminates the need for complex automations by relying on simple PDF files as the universal format.
Summary
The video presents a personal knowledge management (PKM) system built around a single local folder and Claude AI, with a focus on integrating handwritten notes and sketches into this system. The creator explains that after testing various tools including reMarkable, Apple Notes, Notability, and GoodNotes, he settled on PDF Expert as the simplest solution because it natively works with PDF files, eliminating the need for conversion steps or complex automations.
The workflow is structured around two key folders inside PDF Expert synced via iCloud: a 'Team Inbox' where the user deposits handwritten notes or sketches for the AI to process, and a 'Needs Feedback' folder where the AI places documents for the user to review and annotate. The AI orchestrator, named Larry, monitors these folders and acts on their contents when instructed, reading PDFs directly and interpreting handwriting and sketches without any OCR preprocessing.
Two primary use cases are demonstrated. First, the creator shared an AI-generated plan as a PDF into the Needs Feedback folder, annotated it with checkboxes and handwritten notes using his Apple Pencil, and then had Claude read the annotations to understand his decisions. Claude successfully identified 15 annotations across 6 pages using PDF metadata, correctly interpreted most handwritten feedback, and updated its behavior via a new 'Handwritten Collaboration Loop' SOP stored in the team knowledge base. Second, the creator sketched a rough layout for a membership profile page and dropped it into the Team Inbox. Claude correctly interpreted the sketch — identifying the header, avatar placement, bio section, and activity panels — and generated a polished HTML/PDF mockup that was then sent back for review and further annotation.
The creator emphasizes that this approach scales well, noting he has run up to 12 parallel AI sessions inside the same folder. He argues that expectations for AI handwriting accuracy should be calibrated realistically, comparing it favorably to human assistants working with poor handwriting. The system is described as personal and private, not meant for team sharing, and represents the culmination of a journey through tools like Evernote, Notion, Obsidian, and Heptabase, all now replaced by a local folder and Claude.
Key Insights
- The creator rejected reMarkable, Apple Notes, Notability, and GoodNotes specifically because they do not natively store content as PDF files, requiring conversion steps or third-party sync hacks that introduce fragility into the workflow.
- The creator found that Claude itself performed OCR on handwritten PDFs without any preprocessing by PDF Expert, meaning the AI handled handwriting recognition natively as part of reading the file.
- The creator argues that AI handwriting accuracy expectations should be lowered to match realistic human assistant benchmarks, noting that a human assistant given the same poorly-written, context-free annotated PDF would likely not perform better than Claude did.
- Claude detected that the annotated PDF had increased in file size compared to the original, using this metadata signal to infer that annotations had been added before even reading the content — identifying 15 annotations across 6 pages from metadata alone.
- The creator describes running up to 12 parallel Claude sessions simultaneously inside the same local folder, handling tasks ranging from tax calculations to coding and support tickets, all coordinated through shared team knowledge files like SOPs and workstreams.
Topics
Full transcript available for MurmurCast members
Sign Up to Access