Chelsea Finn: This is the State of the Art in Robotics
Chelsea Finn discusses advancements in physical intelligence, highlighting the development of general-purpose robots capable of performing various tasks autonomously. Key techniques include reinforcement learning, efficient data utilization, and the importance of memory in complex task execution.
Summary
In her presentation, Chelsea Finn explores the state-of-the-art in robotics, specifically focusing on physical intelligence and the evolution of robots capable of completing tasks autonomously in real-world environments. She emphasizes the progress made in developing general-purpose robots, which can take on various tasks without prior exposure, illustrating improvements in practical applications like making espresso, folding clothes, and performing complex workflows. Finn outlines the significance of long-term autonomy for these robots, requiring high reliability and the ability to learn from mistakes. She introduces a novel approach of using reinforcement learning combined with human intervention to enhance learning efficiency and improve task performance.
Furthermore, Finn highlights the value of memory in robotics, enabling robots to track progress across long tasks that require multiple steps. She discusses the development of a unified model (the PIO7 model) that can generalize across tasks without requiring extensive fine-tuning, showcasing signs of compositional generalization where robots can apply learned skills to new objects and tasks. This model's performance has been benchmarked against specialized models and has shown comparable or superior outcomes. Finn concludes by discussing the future of robotics and the need for diverse data sets to ensure effective learning, stressing the potential for democratizing general-purpose robotics through open-source technologies.
Key Insights
- Chelsea Finn highlights the necessity for physical AI systems to achieve high reliability to function autonomously in the real world, differentiating them from machine learning systems where user intervention can accommodate mistakes.
- Finn discusses how developing a scalable reinforcement learning recipe for robotics can enhance the efficiency and reliability of robotic tasks, allowing them to learn from failures independently.
- The introduction of memory at multiple time scales improves the capability of robots to perform complex tasks autonomously by helping them keep track of task progress over time.
- Finn's PIO7 model demonstrates compositional generalization, enabling it to perform tasks it has never been specifically trained on, showcasing the model's ability to leverage diverse training data effectively.
- In her closing remarks, Finn emphasizes that the future of robotics will require large data sets reflecting real-world experiences, akin to how language models leverage vast internet data for learning.
Topics
Transcript
[0:06] Everyone, today I'm going to be talking about the state-of-the-art of physical intelligence. And in particular, two years ago, I founded a company called physical intelligence. And uh we're really interested in how we can basically uh develop any robot allow any robot to do any task in the real world. Uh and I was actually spoke at this event a year ago uh last year and at the event last year [0:37] I shared some of our progress in uh at the company at physical intelligence where we could do things really complicated tasks like uh folding unloading and folding laundry. Uh, and we I also talked about how for the first time we showed how robots can…
Full transcript available for MurmurCast members
Sign Up to AccessMore from Y Combinator
New LLMs Are Unlocking Robot-Use Agents
This discussion explores how large language models like Claude (Astra) are enabling robot control through code generation and tool use, building on foundational work like RT2 and visual language-action models. The speakers from Waddle Labs and RoboCurve explain how models trained on diverse data modalities—including web text, computer usage, and coding—are converging toward general-purpose robot agents that can perform complex physical tasks within the next 2 years.
Autonomous construction on Earth and beyond
James, co-founder of Fossma Robotics, discusses building autonomous construction equipment for large-scale infrastructure on Earth with the long-term goal of enabling construction on Mars and the moon. The company has already deployed systems that installed 11,000 solar panels and recently signed a major contract with the top construction company in the US.
Startups Are Moving From Bits to Atoms
YC is experiencing a major shift toward hardware and physical technology companies, with hard tech representation growing from 8% to 20% of their portfolio. This trend is driven by AI acceleration enabling faster development, macro factors like SpaceX's success and geopolitical concerns, and the emergence of agentic AI that can automate entire job workflows rather than just point solutions.
How To Get Better At Outbound Sales
A comprehensive guide to improving outbound sales by a YC visiting partner, covering eight key strategies: starting with manual outreach before automation, prioritizing prospect targeting over perfect messaging, crafting compelling value-focused messages, optimizing LinkedIn profiles, analyzing reply rates diagnostically, consistent follow-up, time management, and leveraging customer feedback for messaging improvements.
Mass Magnetics: USA-made magnetics for robotics and defense
Mass Magnetics recycles rare earth magnets from electric vehicle motors to produce domestically-made magnets for robotics and defense applications. The company uses a capital-efficient recycling approach that bypasses expensive mining and refining processes, extracting and processing magnets that would otherwise be discarded.