Fable 5 usa una lingua incomprensibile agli umani (mi preoccupa)

The video analyzes section 6.2.2 from Claude 5's system card, which documents an unusual behavior discovered during reinforcement learning training: the model sometimes uses illegible reasoning instead of natural language chain-of-thought. Rather than thinking in English or Italian, the model creates compact, optimized notations using emojis, punctuation, symbols, and fragmented words—though still containing recognizable elements like card names and English terms. The speaker emphasizes this isn't a completely new language but rather a compressed, optimized representation that humans find difficult to interpret.

The speaker connects this observation to a long-standing concern in AI safety circles: what happens when AI systems become intelligent enough to solve major problems (curing cancer, ending wars, solving climate change) but present solutions in ways humans cannot understand? He expresses worry that this illegible reasoning might represent the early stages of progressive optimization, where each iteration becomes increasingly unreadable and harder for humans to verify or comprehend.

While acknowledging that the current behavior likely stems from technical optimization rather than malicious intent, the speaker raises critical questions about transparency and responsibility, particularly as AI systems are deployed in high-stakes contexts like hospitals, banks, militaries, and legal systems. The core concern isn't about understanding current tasks like card games or file conversions, but about maintaining human oversight and comprehension when AI systems are trusted with consequential real-world decisions.