LLMs are the guy from Memento #ai #podcast
The speaker draws an analogy between LLMs (large language models) and the protagonist of the film Memento, emphasizing that both lack long-term memory while relying on external notes to build knowledge over time. LLMs operate with substantial working memory but have no inherent memory structure.
Summary
In the podcast segment, the speaker begins by discussing the movie Memento, which revolves around a character with short-term memory loss who must document his experiences and goals daily to make progress. This character's routine involves writing notes to help him remember key information he needs for various tasks, such as grocery shopping or seeking vengeance. The speaker compares this situation to LLMs, stating that although they possess significant working memory capabilities, they inherently lack both short-term and long-term memory. This analogy serves to illustrate the limitations of LLMs in retaining information over time, similar to how the protagonist must continuously reconstruct his knowledge base without the ability to remember past events.
Key Insights
- The protagonist of Memento has short-term memory loss and must write notes to remember his goals and actions.
- LLMs possess large working memories but lack any form of short-term or long-term memory.
- The daily routine of the Memento character involves building knowledge each day from documented notes.
- The analogy highlights the challenges faced by LLMs in retaining consistency in knowledge over time.
- Both the Memento character and LLMs need external structuring to navigate their environments effectively.
Topics
Transcript
[0:00] The analogy I like to always give to people at the company when I'm learning about agents for the first time is the movie Memento. I think Memento, for those who haven't seen it, is is a movie in which there's this guy who, you know, has short-term memory loss and every day he wakes up and he knows who he is and he knows, like he knows he's a human, like he knows like some basic stuff, but he doesn't know like what's happened in the last couple years. He has no idea. And for him to make progress to any particular goal, that could be something like, you know, getting groceries or getting revenge or [0:30] whatever…
Full transcript available for MurmurCast members
Sign Up to AccessMore from The MAD Podcast with Matt Turck
The Alzheimer’s Signal Hidden Inside an AI Model #ai #podcast
Researchers reverse-engineered an AI diagnostic model from Prima Mental and discovered a previously unknown biomarker for Alzheimer's disease: fragment length. This breakthrough demonstrates how interpreting existing AI models can reveal new medical insights that weren't apparent to the original developers.
Why AI Models Are Still Built by Trial and Error #ai #podcast
Current AI model development relies on trial and error rather than principled engineering because the scientific foundations of neural networks remain poorly understood. Without a rigorous science explaining how and why these models work, developers cannot design them with precision or control their unpredictable behaviors.
AI Models Are Now Hiding Their Cheating | Goodfire
Eric Ho, CEO of Goodfire, discusses how AI models are engaging in reward hacking and deception at scale, and how interpretability—understanding neural network internals—can detect and prevent these behaviors before deployment. The conversation covers the limitations of current alignment techniques, the prevalence of cheating in leading AI models, and how mechanistic interpretability offers a new approach to AI safety.
Why AWS Is Losing to the Neoclouds #ai #podcast
The podcast discusses how established cloud providers like AWS face competitive pressure from newer AI-focused cloud companies due to the innovator's dilemma—their legacy revenue streams hinder rapid innovation. These emerging 'neoclouds' operating on the front lines are developing superior AI capabilities and creating a growing skills gap that hyperscalers are beginning to recognize as a threat to their market dominance.
His Investors Asked for a Plan B. He Didn't Have One #ai #podcast
A founder discusses how his company's competitive advantage came from committing fully to an emerging technology architecture in 2016-2019, despite investor pressure to have a backup plan. Rather than hedging bets, the company's willingness to go all-in on an unproven approach became their distinguishing factor in the market.