Why Robotics Still Isn't Solved - But Could Be Soon | YC Paper Club

The YC Robotics Club gathered to discuss the persistent gap between robotics hype cycles and actual capabilities. The opening speaker noted this is the 10th year of predictions that robotics will be solved, dating back to AlphaGo. Despite impressive recent demonstrations (Aloha for teleoperation, diffusion policies, VLMs), core unsolved problems remain.

Four fundamental walls blocking progress were identified: (1) Physical real world modeling—video prediction models fail to respect physics and don't bridge the sim-to-real gap; (2) Action conditioning—models struggle when conditioning on actions, requiring massive data to learn the ST+1 transition function; (3) Sensorimotor feedback—robots lack the rich tactile sensing humans possess (detecting normal force, tangent force, friction coefficients), limiting world model learning; (4) Embodiment drift—actuator degradation, corrosion, and battery decay change robot dynamics over time, requiring continuous retraining.

Marcel presented Multiscale Embodied Memory (MAM), addressing long-horizon task execution through hierarchical memory. High-level policies track task progress using compressed text-based memory while low-level policies maintain dense visual frames for dextrous control. This architecture enabled tasks lasting tens of minutes, including in-context adaptation where robots recover from mistakes mid-execution—a capability lacking in standard policies.

Milan discussed self-supervised bootstrapping for embodied reasoning, proposing that reasoning type should be discovered rather than imposed uniformly. The R3B approach treats reasoning as a latent variable, using variational inference to identify task-relevant reasoning patterns (e.g., move-and-gripper reasoning for manipulation, affordance reasoning for legged locomotion). Results showed selective reasoning outperforms exhaustive reasoning across embodiments.

Tyler presented Sim-to-Real for tool use, training a single policy via RL in GPU-accelerated simulation to perform diverse dextrous manipulation across novel tools. Rather than learning from teleoperation, the system generates decades of interaction data in days, enabling zero-shot generalization to unseen tools and tasks by conditioning policies on goal pose sequences extracted from human videos. Play-to-Perfect extends this to precise assembly through pre-training on task-agnostic play.

Nico argued that the robotics industry should shift toward robotics application companies—businesses solving specific customer problems end-to-end using existing hardware and starting with teleoperation, rather than pursuing general foundation models. Success requires fast iteration: replica customer environments for testing, repeatable evaluation metrics encoding business-specific requirements, and efficient data collection with continuous training-evaluation loops. He emphasized that physical data's multimodal, episodic nature requires specialized infrastructure different from web data systems.

General Instinct presented optimizations for world action models (WAMs), which predict future frames and kinematics simultaneously to learn physics-grounded action policies. While WAMs outperform VLMs, standard implementations are prohibitively slow (2 GB GPU memory, $70k hardware per unit). Their optimizations—distilling diffusion steps from 50 to 1-2, using cross-attention between video and action transformers instead of explicit frame decoding, and asymmetrical denoising—achieved 500ms inference per 16-frame chunk on consumer GPUs, making deployment economically viable.