Chelsea Finn: This is the State of the Art in Robotics
Chelsea Finn discusses advancements in physical intelligence, highlighting the development of general-purpose robots capable of performing various tasks autonomously. Key techniques include reinforcement learning, efficient data utilization, and the importance of memory in complex task execution.
Summary
In her presentation, Chelsea Finn explores the state-of-the-art in robotics, specifically focusing on physical intelligence and the evolution of robots capable of completing tasks autonomously in real-world environments. She emphasizes the progress made in developing general-purpose robots, which can take on various tasks without prior exposure, illustrating improvements in practical applications like making espresso, folding clothes, and performing complex workflows. Finn outlines the significance of long-term autonomy for these robots, requiring high reliability and the ability to learn from mistakes. She introduces a novel approach of using reinforcement learning combined with human intervention to enhance learning efficiency and improve task performance.
Furthermore, Finn highlights the value of memory in robotics, enabling robots to track progress across long tasks that require multiple steps. She discusses the development of a unified model (the PIO7 model) that can generalize across tasks without requiring extensive fine-tuning, showcasing signs of compositional generalization where robots can apply learned skills to new objects and tasks. This model's performance has been benchmarked against specialized models and has shown comparable or superior outcomes. Finn concludes by discussing the future of robotics and the need for diverse data sets to ensure effective learning, stressing the potential for democratizing general-purpose robotics through open-source technologies.
Key Insights
- Chelsea Finn highlights the necessity for physical AI systems to achieve high reliability to function autonomously in the real world, differentiating them from machine learning systems where user intervention can accommodate mistakes.
- Finn discusses how developing a scalable reinforcement learning recipe for robotics can enhance the efficiency and reliability of robotic tasks, allowing them to learn from failures independently.
- The introduction of memory at multiple time scales improves the capability of robots to perform complex tasks autonomously by helping them keep track of task progress over time.
- Finn's PIO7 model demonstrates compositional generalization, enabling it to perform tasks it has never been specifically trained on, showcasing the model's ability to leverage diverse training data effectively.
- In her closing remarks, Finn emphasizes that the future of robotics will require large data sets reflecting real-world experiences, akin to how language models leverage vast internet data for learning.
Topics
Transcript
[0:06] Everyone, today I'm going to be talking about the state-of-the-art of physical intelligence. And in particular, two years ago, I founded a company called physical intelligence. And uh we're really interested in how we can basically uh develop any robot allow any robot to do any task in the real world. Uh and I was actually spoke at this event a year ago uh last year and at the event last year [0:37] I shared some of our progress in uh at the company at physical intelligence where we could do things really complicated tasks like uh folding unloading and folding laundry. Uh, and we I also talked about how for the first time we showed how robots can…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Y Combinator
Circleback CEO Ali Haghani: Recording Company Meetings Will Become The Norm
Ali Haghani discusses CircleBack, an AI note-taking tool designed to capture, organize, and automate meeting content. He shares insights on his hardware setup, unexpected uses of CircleBack, and the evolving landscape of software engineering as AI capabilities grow.
Why Robotics Still Isn't Solved - But Could Be Soon | YC Paper Club
YC Paper Club discusses why robotics remains unsolved despite repeated predictions of breakthrough years, presenting four fundamental challenges (physical modeling, deformable objects, sensorimotor feedback, embodiment drift) and showcasing recent advances in memory-augmented policies, embodied reasoning, sim-to-real learning, and world action models.
Max Hodak: What Really Kills Deep Tech Startups?
Max Hodak, CEO of Science (a retinal prosthesis company), explains that deep tech startup success is determined by infrastructure and iteration speed rather than technical brilliance. He emphasizes that proper systems for purchasing, recruiting, and performance management are as critical as the core technology, and shares specific processes Science uses to scale efficiently.
Building the First Data Centers in Space
Philip Johnston, CEO of StarCloud, discusses building data centers in space to address Earth's energy bottleneck for AI. The company launched its first satellite (StarCloud 1) in November 2025 with an Nvidia H100 GPU, and has raised $170 million to deploy a constellation of 88,000 satellites that could provide 20 gigawatts of compute capacity.
Patrick Collison: "What If You Succeed?"
Patrick Collison discusses Stripe's founding story, the importance of concrete customer problems, and shares data suggesting it's the best time ever to start a business. He argues that despite AI advancement, there will be many winners rather than centralization, and emphasizes the value of deep learning alongside AI tools.