Gary Marcus floats a legal argument worth taking seriously: chatbot output is the company's own speech, not third-party speech, which is the only thing Section 230 was ever built to shield. If US courts followed Germany's lead, every major LLM provider would suddenly own the consequences of its model's fabrications.
A recent German court decision holding a company liable for its chatbot's errors has reopened a question that AI companies would prefer stayed closed: does Section 230 of the Communications Decency Act actually protect them at all?
The argument, raised by Gary Marcus in a post on his Substack and echoed independently by AI law scholar Ryan Calo, rests on a distinction that is easy to state and hard to wriggle out of. Section 230 immunizes "interactive computer services" from liability for content provided by another information content provider. It is, and always has been, about third-party speech. The classic case is an internet provider or platform carrying something a user posted. The platform did not write it, so the platform is not on the hook for it.
A large language model is a different animal. When Google's or OpenAI's system generates a sentence that defames a real person, no third party wrote that sentence. The model did. The company built, trained, and shipped the model, and the output is a product of that software. Under this reading, chatbot output is the company's own speech, and Section 230's third-party shield simply does not reach it.
What's claimed
The broad claim circulating is that AI companies enjoy something like blanket protection from liability for what their models say. That has always been more assumption than settled law. Section 230 has never been tested at the Supreme Court level against generative output, and the statute's text was written in 1996 for a web of message boards and hosting providers, not for systems that synthesize novel text on demand.
What's actually new
The German ruling is the concrete development. A court there held a company responsible for false statements its chatbot produced, treating the output as the company's responsibility rather than some unattributable third-party utterance. That framing maps cleanly onto the structure of US Section 230 doctrine. If American courts reasoned the same way, the analysis would not require new legislation. It would just require recognizing that model output is first-party speech, which is what the German decision effectively says.
There is a separate legislative track running in parallel. A bipartisan group of senators, including Lindsey Graham, Dick Durbin, Josh Hawley, Amy Klobuchar, Richard Blumenthal, and Marsha Blackburn, introduced a bill to sunset Section 230 entirely. GovTrack gives that effort roughly a 1% chance of enactment, and given the funding behind the opposition, even that may be generous. The point of the German-inspired argument is that the courts could get to a similar place without Congress acting at all.
Why it matters for the technology
This is where the legal question becomes a technical one. Current LLMs fabricate as a matter of course. Hallucination is not an occasional bug that better prompting eliminates; it is a structural property of next-token prediction over a probabilistic model with no grounded notion of truth. Models invent citations, attribute crimes to real people, and dispense confidently wrong medical and legal information. Marcus cites the well-documented case of a law professor falsely accused of harassment by a model that simply made the accusation up, which he wrote about in Taming Silicon Valley.
If providers were held liable for that output the way a publisher is liable for what it prints, the economics of shipping a system that fabricates "fairly regularly" would change overnight. You cannot ship a product that exposes you to defamation suits every time it confabulates a plausible-sounding falsehood. That pressure would push the industry toward architectures with stronger grounding, retrieval that is actually verified against sources, and abstention behavior when confidence is low. The liability question and the reliability question are the same question wearing different clothes.
Limitations
A few caveats keep this from being a done deal. First, the German decision is not binding on anything in the US; it is a persuasive analogy, not precedent. American courts have been generous to platforms under Section 230 for decades, and there is no guarantee they would adopt the first-party-speech framing even though the text arguably supports it.
Second, there is an active push in the other direction. The same companies that talk about responsible AI have lobbied for protective state legislation. Marcus points to industry support for an Illinois proposal that would shield AI labs from liability even in cases involving mass-casualty harms, defined as death or serious injury to 100 or more people or at least $1 billion in property damage. That is roughly the opposite of accepting product responsibility, and it shows where the lobbying energy is actually going.
Third, the first-party framing is cleaner in defamation and false-statement cases than in others. A chatbot that summarizes or quotes a user's input occupies murkier ground, and defendants will argue every output is somehow user-elicited third-party content. The strength of the German analogy is precisely that pure fabrication, content the model originated with no third-party source, is the hardest case for the third-party shield to cover.
The larger pattern here is familiar. A technology ships fast under the assumption that an old legal shield covers it, the assumption goes untested for years, and then a single ruling somewhere forces everyone to check whether the shield was ever the right shape. The German court did not write new AI law. It applied an old principle, that you answer for what your own product says, and that principle has an obvious home in US law if a court decides to put it there.
Comments
Please log in or register to join the discussion