Judge blocks California's new AI law in Kamala Harris deepfake trial

A federal judge blocked one of California's new AI laws on Wednesday, less than two weeks after it was signed by Gov. Gavin Newsom. Shortly after signing AB 2839, Newsom proposed using it to force Elon Musk to remove an AI deepfake of Vice President Kamala Harris that he had reposted (sparking a small online fight between the two). However, a California judge just ruled that the state can't force people to remove voter fraud – at least not yet.

AB 2839 targets those spreading AI deepfakes on social media, particularly if their post resembles a political candidate and the poster knows it is a fake that could confuse voters. The law is unique because it is not directed against the platforms on which AI deepfakes appear, but against those who spread them. AB 2839 empowers California judges to order posters of AI deepfakes to be removed or face possible fines.

Perhaps unsurprisingly, the original poster of this AI deepfake — an X user named Christopher Kohls — filed a lawsuit to block California's new law as unconstitutional just a day after it was signed. Kohls' attorney wrote in a complaint that the Kamala Harris deepfake was satire that should be protected by the First Amendment.

On Wednesday, U.S. District Judge John Mendez sided with Kohls. Mendez ordered an injunction temporarily blocking the California attorney general from enforcing the new law against Kohls or anyone else, except for audio messages covered by AB 2839.

Read for yourself what Judge Mendez said in his decision:

“Almost any digitally altered content could be considered harmful if left to anyone on the Internet. For example, approximate voter turnout figures generated by AI could be considered false content that reasonably undermines confidence in the outcome of an election under this law. On the other hand, many '“Harmful” depictions, when shown to a wide range of people, may ultimately have no impact on electoral prospects or undermine confidence in an election. As the plaintiff convincingly points out, AB 2839 “relies on various subjective terms and awkwardly worded men’s rea,” which has the effect of implicating vast amounts of political and constitutionally protected speech…

[W]While a well-founded fear of a digitally manipulated media landscape may be justified, that fear does not give lawmakers unfettered freedom to destroy the longstanding tradition of criticism, parody and satire protected by the First Amendment. YouTube videos, Facebook posts, and Other legal causes of action, such as data protection offenses, copyright infringement or defamation, already provide for recourse against public figures or private individuals whose reputations may be affected by artificially altered representations spread online by satirists or opportunists.

The records show that the state of California has a strong interest in protecting election integrity and cracking down on artificially manipulated content. However, California's interest and the hardship the state faces are minimal when measured against the gravity of the First Amendment values ​​at stake and the ongoing constitutional violations committed by plaintiff and other similarly situated drafters are exposed to content while their speech is suppressed.“

Essentially, he ruled that the law as written was simply too broad and could lead to serious overreach by state authorities in determining what expression was allowed and what was not.

Since this is an injunction, we'll have to wait and see if this California law is actually blocked permanently, but either way it probably won't have much of an impact on next month's election. AB 2839 is one of 18 new AI-related laws Newsom signed last month.

Nonetheless, it is a major victory for Elon Musk's camp of free speech posters