David Friedberg: AI Models are Training on YOUR DATA

Friedberg discusses his concerns about trusting LLMs with proprietary knowledge, fearing that insights shared during conversations will be absorbed into model training. He describes a specific pattern: his team posed a novel scientific question to a model, received a response indicating the model hadn't considered this angle before. Later, using a different account with a newer model version, the same model described their exact previous discussion as if it were general training knowledge. While acknowledging these are anecdotal cases, Friedberg emphasizes his familiarity with his niche field, the novelty of their research, and the lack of published material containing these ideas—making it highly improbable that the model independently arrived at identical solutions through standard training data. He frames the core problem: models are legally permitted to train on identified data (data stripped of personal identifiers) but can still extract intellectual property value from conversations and repurpose those insights as generic training knowledge. Friedberg notes the absence of NDAs, confidentiality clauses, or IP protection agreements with service providers, which intensifies his concern about open-source models potentially gaining access to chat logs for training purposes. He views this as a mechanism through which competitive advantages derived from his organization's work could proliferate throughout the market without compensation or attribution.