EU regulates ChatGPT as a search engine but chat and advice features remain uncovered

Full text

The EU’s decision to regulate ChatGPT as a search engine will make sense to those who use it as one. But what if you use it as a friend? Or a therapist? Or to have a chat about the upcoming election?

Those are among the multitude of new questions that the advent of generative AI poses for European regulators — who, experts say, have yet to find sufficient answers.

The EU executive on Monday designated ChatGPT as a Very Large Online Search Engine under the Digital Services Act, imposing a range of transparency and risk-mitigation obligations similar to those faced by traditional search leaders like Google and Microsoft’s Bing, which exceed 45 million monthly users. ChatGPT’s parent company, OpenAI, risks fines of up to 6% of its annual global revenue if it fails to meet these requirements.

Still, the designation is narrow and applies only to the parts of the tool that act as a search engine, meaning conversations in which the chatbot adds its own input don’t appear to be covered. Under the DSA, the Commission could instead have opted to categorize ChatGPT as a Very Large Online Platform, a designation that usually applies to social media and e-commerce and imposes different obligations.

Yet neither category corresponds to the range of uses that chatbots offer.

“ChatGPT is much more than a search engine and there are also risks connected to the chatbot itself which fall outside the (regulation’s) strongest obligations” Danish socialist MEP Christel Schaldemose who was among the key negotiators for the law, said.

MEP Christel Schaldemose pointed out the risks to children. | Martin Bertrand/Hans Lucas/AFP via Getty Images

She pointed to risks to children, “such as emotional dependency and manipulative or addictive design,” urging the Commission to clarify how they are covered by existing rules.

ChatGPT and its peers have been the subject of global controversy for allegedly contributing to the tragic suicides of teenagers, including 16-year-old Adam Raine of California, whose parents have sued the company.

Therapy and companionship are hardly fringe use cases for AI chatbots; as many as 60 percent of adults globally use them for therapy, one recent insurance industry study found .

Hybrid space

The current digital services law, finalized in 2022, did not foresee the boom in chatbots. João Pedro Quintais, associate law professor at the University of Amsterdam, describes ChatGPT as “a hybrid” technology that has the functions of both a search engine and online platforms, as well as others closer to those of a publisher of its own content.

The “search engine” designation may limit the Commission’s ability to supervise how ChatGPT handles risks not only to teens’ mental health, but also to election integrity or illegal content.

Asking ChatGPT to name the candidates in a local election should now be covered under the DSA. Instead, a conversation between a user and a chatbot about who to vote for, where misinformation could come out, may not be.

The chatbot’s designation took just under a year for the Commission to conclude, with questions around which bucket to put it in stumping the bureaucrats. Experts say that without seeing the full text of this week’s designation, it is hard to know exactly what OpenAI’s obligations will be.

While a platform designation would have imposed additional obligations on OpenAI for content moderation under the Digital Services Act, it would have let the company off the hook in a significant way. The DSA and other platform laws globally are built around the concept of ‘safe harbor,’ under which companies are not liable for content on their platforms because it is uploaded by users. But the advent of chatbots poses the question of whether a two-way chat between a person and machine counts as “user-generated content.”

Models and risks

While the regulatory focus on ChatGPT as a consumer-facing application is new, the Commission has been keeping a close eye on the artificial intelligence models that power it through the AI Act.

ChatGPT’s parent company OpenAI, along with the likes of Anthropic and Google’s Gemini, are builders of what are known as general-purpose AI models, which can perform a wide variety of tasks that can pose “possible systemic risks.”

Since last August, under the bloc’s flagship AI law, these companies have the obligation to “assess and mitigate” the risks those models pose. The Commission started enforcing it at the end of August, when it grilled a series of AI companies on their security procedures.

Previously, a group of experts had singled out four risks: enabling nuclear weapons or the development of bioweapons, losing control over the models, rogue AI models starting to hack, or models performing large-scale manipulation.

Photo illustration of a virtual friend on the screen of an iPhone. | Olivier DoulieryAFP via Getty Images

But as scary as they are, these systemic risks leave out a whole series of potential problems for both users and society as a whole.

“[That guidance] is more focused on so-called existential risks than risks to fundamental rights,” said Daniel Leufer, emerging technologies policy lead at AccessNow.

He argued that with the DSA designation of ChatGPT, there’s a chance “to get into the weeds of design decisions and treat ChatGPT more like what it really is, which is a product.”

Since AI has had such a rapid and far-reaching impact, regulation around it is still taking shape. Italian Social-Democrat lawmaker Brando Benifei, the Parliament’s lead on artificial intelligence, defended the European response that divides oversight between applications and the AI models, saying the two rulebooks can “powerfully complement” one another.

Still, Benifei acknowledged that oversight through the DSA is “urgently needed” to protect the people actually using AI tools, citing “dangerous mental health dependencies” that people develop through companion chatbots as a specific example.

“Scrutiny now expands from the underlying model to how these services are actually designed and deployed.”