The Court Documents Unveiled
In a surprising turn, newly released court filings from the 2023 lawsuit filed by The New York Times (NYT) against OpenAI and Microsoft have surfaced. The documents, part of the ongoing litigation, contain internal communications that paint a stark picture of how the two tech giants approached the training of large‑language‑model (LLM) systems. Microsoft executives, including Dr. Brent Hecht, director of Applied Science, described OpenAI’s web‑scraping as the “largest theft of labor in human history,” while OpenAI’s Nick Turley called the practice an “existential threat to publishers.” These statements, once confined to private memos, now form a public record that could influence the trajectory of AI development and media economics.
The lawsuit, originally filed alongside five other writers, alleges that OpenAI and its partners scraped millions of news stories from the internet without permission or compensation. The court documents confirm that the data collection involved bypassing paywalls and systematically removing copyright notices from the scraped text. The NYT case is now a pivotal battleground for determining whether AI companies can invoke “fair use” doctrines—such as parody or journalism—to justify training on copyrighted material.
Why It Matters to Journalism and AI
The Threat to the Supply Chain
The internal memo from 2023, quoted in the filings, warned that large AI models “are a product that destroys its supply chain.” For publishers, the supply chain is the flow of original content to readers. If AI models can generate convincing text without paying for the original, the incentive to produce new journalism diminishes. This could lead to a “doom loop,” where fewer original stories are written, and AI models become increasingly reliant on the very content they are trained on.
Fair Use Under Siege
The legal question hinges on whether the training of LLMs constitutes a transformative use that falls under fair use. The NYT’s argument is that the AI’s use of copyrighted text is not transformative enough to justify the absence of licensing fees. Conversely, Microsoft’s spokesperson Alex Haurek has stated that Copilot, an AI tool integrated into Microsoft Office, is “not a substitute for publishers’ journalism.” However, the court documents suggest that the same underlying training data that powers Copilot and Chat GPT was derived from the same scraped corpus, raising doubts about the distinction.
Industry Ripple Effects
If the court rules against OpenAI and Microsoft, it could force a reevaluation of how AI companies source data. Publishers might demand licensing agreements, and new regulatory frameworks could emerge. The ripple effect would touch not only media companies but also the broader AI ecosystem, where data is the lifeblood of model performance.
Technical Breakdown of the Scraping Process
The unredacted court materials provide a granular view of the scraping methodology:
- Paywall Bypass: The documents detail how OpenAI’s data collection pipeline accessed articles behind subscription barriers by exploiting public APIs and caching mechanisms that were not intended for large‑scale harvesting.
- Massive Scale: The dataset reportedly included millions of documents, spanning a wide range of topics and publication dates. This breadth is essential for training models that can generalize across domains.
- Copyright Notice Removal: Once the text was extracted, the pipeline stripped metadata, including copyright notices and author attribution. This step effectively anonymized the source material, making it difficult to trace back to the original publishers.
The removal of copyright notices is particularly troubling from a legal standpoint. Copyright law protects not only the text but also the expression of ideas. By erasing these notices, the data set becomes a sanitized pool that can be used without clear attribution or licensing, which is precisely what the NYT alleges.
Industry Impact and Legal Precedents
Several similar lawsuits have already ruled in favor of AI companies, often citing the transformative nature of the AI’s output. However, judges have consistently noted that the law governing AI use remains unsettled. The NYT case is therefore a potential turning point. A ruling against OpenAI and Microsoft could:
- Mandate Licensing: Publishers may be required to negotiate licensing agreements with AI firms, creating a new revenue stream for media outlets.
- Encourage Data Governance: AI companies might adopt stricter data governance protocols, including explicit consent mechanisms and transparent data provenance.
- Influence AI Development: Developers may need to pivot toward synthetic data generation or open‑source datasets that are explicitly licensed for training.
The broader tech community is watching closely. For instance, the recent launch of Anthropic’s physical biology lab demonstrates how AI research is expanding into new domains, raising similar questions about data sourcing and ethical use. Likewise, consumer products like the Samsung Galaxy S26 FE illustrate how hardware and software ecosystems can coexist with AI services, yet the legal frameworks governing data use remain ambiguous.
Read the full breakdown originally published at https://ltdeveloperblogs.github.io/posts/microsoft-executive-called-openais-web-scraping-the-largest-theft-of-labor-in-human-history/
