Back to Learning Area

Data Protection and AI: When the GDPR Meets Neural Networks

The GDPR and language models — a relationship that needs clarification.

  • Ethics
  • 10 min
  • Intermediate

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Data protection and AI

What is the GDPR?

In Europe, there is an important law. It is called GDPR. GDPR is the EU's data protection law. It protects your personal data.

GDPR and AI

AI programs learn from lots of data. AI means Artificial Intelligence. This is a conflict with the GDPR. The GDPR says: only as much data as necessary. This is called data minimization.

Your rights

You have rights regarding data protection. You can demand: delete my data. This is called the right to be forgotten. Companies must follow this.

Italy halted ChatGPT in March 2023 — not because AI is illegal in Europe, but because the Italian data protection authority accused OpenAI of having no valid legal basis for processing the data of millions of people. The GDPR does not forbid AI. It forbids careless data handling.

Two GDPR principles — data minimization and purpose limitation — together with the right to be forgotten create genuine technical conflicts with how modern AI systems are built. This article unpacks the three central flashpoints and explains what rights you have as a citizen.

Data Minimization vs. Scaling Laws

Definition

Data Minimization (Art. 5(1)(c) GDPR)

Analogy

Think of a fishing trawler: it drags a massive net across the ocean (the internet), catching everything — including protected species (personal data). The GDPR says: "You may only fish with a rod, catching exactly what you need." The tension: trawling is more efficient, but rod fishing respects the ecosystem.

The analogy suggests the only solution is to stop trawling. In reality, technical measures like anonymization, synthetic data, or differential privacy can partially reconcile the two approaches.

AI Training

More data means a better model. Large language models need billions of texts — inevitably containing personal data.

GDPR

Only the minimum necessary data may be processed. Every processing operation requires a legal basis and a defined purpose.

Purpose limitation makes this harder: a fitness app collects your data to analyze your workouts — not to train an AI model. Data may only be reused for a new purpose if that purpose is compatible with the original one (Art. 6(4) GDPR) or if a separate legal basis applies. For health data, Art. 9 adds a further hurdle, and often only explicit consent remains. Pseudonymization helps, but it changes nothing for the provider itself: whoever can link the data back to the individuals is still processing personal data. Only for a recipient who realistically cannot re-identify the individuals can the data lose its personal character, according to a 2025 ruling of the Court of Justice of the European Union.

So which legal basis covers training? Consent is hardly practical for billions of texts from the web. The research privilege (Art. 89 GDPR) eases reuse for scientific purposes, but it is not a legal basis in itself. In practice, it usually comes down to legitimate interest (Art. 6(1)(f)). For a long time it was unclear whether that holds for AI training — by now the line is clearer. In its Opinion 28/2024 of December 2024, the European Data Protection Board (EDPB) considers it possible if a three-step test is passed case by case: the provider pursues a legitimate interest, the processing is necessary for it, and the interests of the data subjects do not override it — which also depends on whether they could reasonably expect such use. Along these lines, the Higher Regional Court of Cologne in May 2025 rejected an urgent application by consumer advocates against Meta training on public Facebook and Instagram posts. In July 2026, the EDPB also put guidelines on web scraping for generative AI out for public consultation, which spell out this balancing test further with examples. The European Commission additionally wants to anchor legitimate interest for the development and operation of AI explicitly in the GDPR (proposal for a "Digital Omnibus" on data and the GDPR of November 2025, new Art. 88c — not to be confused with the AI Omnibus proposed at the same time, which has already amended the AI Act), combined with an unconditional right of data subjects to object. The dispute is over whether this turns into a free pass for providers and how strong the right to object should be. As of November 2026, the proposal has not been adopted: the member states agreed on their negotiating position in October 2026 and dropped the unconditional right to object, leaving the general right to object under Art. 21. The European Parliament has not yet settled on its position. Until an amendment is adopted and enters into force, the GDPR applies in its current form. How quickly things can escalate without a solid legal basis is shown by the Italy case:

  1. 1. March 2023: The data protection authority Garante temporarily halts ChatGPT in Italy — among other things, a legal basis for training is said to be missing
  2. 2. April 2023: OpenAI adds privacy notices, an objection form and an age check — ChatGPT is available again
  3. 3. December 2024: The Garante imposes a fine of 15 million euros — training without a legal basis defined in advance, insufficient information, an unreported data breach, no age verification
  4. 4. March 2026: A court in Rome annuls the decision — not on the merits, but because OpenAI had meanwhile set up an establishment in Ireland, making the Irish authority the lead authoritythe GDPR's one-stop shop

Caution

Common Misconception: "The GDPR bans AI in Europe"

The GDPR does not ban AI — it bans uncontrolled processing of personal data. AI is legal under the GDPR when it is developed with a solid legal basis (such as consent or legitimate interest) and safeguards such as anonymization. Italy did not permanently ban ChatGPT — it required compliance.

The Right to Be Forgotten — A Technical Dilemma

Definition

Right to Erasure (Art. 17 GDPR)

Analogy

Think of training as baking a cake: the training data is the ingredients — eggs, flour, sugar, and someone's personal recipe card that accidentally fell in. Once the cake is baked (the model is trained), you cannot unbake it to remove the recipe card. The ingredients have fused into the final product. Output filters are like putting frosting over the part where the card shows through — it hides it, but it is still inside.

There is a crucial distinction: training data (can be deleted from servers) vs. model weights (information distributed, cannot be surgically removed). One is a file problem; the other is a math problem.

What happens when a person demands that ChatGPT "forget" them? OpenAI can activate an output filter (the name no longer appears in responses) and exclude the person from future training data. But the existing model may still contain the information in its weights — it then simply no longer expresses it. The person may be hidden, not forgotten.

Machine Unlearning is an active research field: how do you make a model specifically forget certain data? Bourtoule et al. (2021) showed an exact approach with their SISA method: the training data is split into shards from the outset, a sub-model is trained per shard, and after a deletion request only the affected shard needs retraining. However, this has to be planned before training and is considered hardly practical for large language models. For models that are already trained, only approximate methods exist, and surveys from 2025 and 2026 reach the same conclusion: these methods mostly just suppress the knowledge — targeted attacks or a little further training can often bring it back. As of November 2026, there is no reliable, verifiable method. The core question remains: how do you prove that a model has really forgotten?

Carlini et al. (2021) showed that hundreds of verbatim text sequences from the training data can be extracted from GPT-2, including names, phone numbers, and email addresses; Nasr, Carlini et al. (2023) managed this on a much larger scale, even with ChatGPT — despite safety training. This underscores: filters hide information, they do not delete it.

Caution

Common Misconception: "Output filters solve the deletion problem"

Output filters prevent the model from displaying information. But the data may remain encoded in the weights and can be extracted under certain conditions. Whether model weights constitute personal data at all was disputed among supervisory authorities — the Hamburg Commissioner for Data Protection and Freedom of Information denied it in a 2024 discussion paper. In December 2024, the EDPB set the line: case-by-case assessment. A model is only anonymous if it is very unlikely that individuals can be identified from it and that their data can be extracted through targeted queries. Filters are a pragmatic workaround, not a legal solution under the GDPR.

A second, related problem: language models sometimes invent false claims about real people. The GDPR requires data to be accurate (Art. 5(1)(d)) and gives you a right to rectification (Art. 16). But a model is about as hard to correct in a targeted way as it is to make it forget. The privacy organization noyb has therefore filed complaints against OpenAI, in Austria in 2024 and in Norway in 2025: there, ChatGPT had falsely described a man as the convicted murderer of his children. No decision by the authorities is known so far (as of November 2026).

Your Rights, Their Obligations

Right of Access (Art. 15) You can find out what data about you is being processed. If an algorithm alone decides about you, you are also entitled to an understandable explanation of how the result came about.
Right to Object (Art. 21) If a provider relies on its legitimate interest, you can object, on grounds relating to your particular situation, to your data being used for training in the future. The provider must then stop, unless it demonstrates compelling legitimate grounds.
Right to Human Review (Art. 22) Decisions with legal or similarly significant effects — such as on loans or job applications — may be made by an AI alone only by way of exception. In that case, you can demand that a human review them.

How seriously the Court of Justice of the European Union takes these rights is shown by two rulings on creditworthiness. In the SCHUFA case (December 2023), it held that an automatically calculated score is itself a decision within the meaning of Art. 22 if the bank relies on it decisively — the credit agency cannot hide behind the fact that, formally, only the bank decides. In the Dun & Bradstreet Austria case (February 2025), it clarified the right of access: anyone assessed automatically is entitled to an understandable explanation of the procedure and principles actually applied. Merely handing over a formula or the algorithm is not enough, and trade secrets do not justify a blanket refusal — in a dispute, the supervisory authority or the court weighs the interests.

Think of a building permit system: before you construct a building (deploy an AI system), you need two permits. One from the building authority (EU AI Act — is the building safe?) and one from the environmental agency (GDPR — are the neighbors' rights respected?). Both must approve before construction begins. Unlike in construction, though, there is usually no authority issuing a permit in advance — the company must check and prove on its own that it complies with both sets of rules.

For a German company deploying a customer service chatbot, this can mean in practice:

  • Conduct a Data Protection Impact Assessment (DPIA) before launch (Art. 35 GDPR) — AI customer support involving personal data is on the mandatory list of the German supervisory authorities
  • Define a legal basis for the chat logs: for handling the request, contract performance or legitimate interest is usually sufficient; if the logs are to serve further purposes, such as training the company's own models, a separate legal basis is needed — often consent. Inform customers in any case (Art. 13)
  • Sign a data processing agreement with the LLM provider (Art. 28 GDPR) — more on this in the decision tree below
  • Remove or pseudonymize personal details where possible before sending them to the LLM
  • Set a retention period and delete the chat logs afterwards, for example after 30 days
  • Label the chatbot as AI — required by Art. 50(1) EU AI Act since 2 August 2026, unless this is obvious anyway

Caution

Common Misconception: "If you have nothing to hide, you have nothing to fear"

Data protection does not primarily protect secrets — it protects the fundamental right to informational self-determination. It is about your power to decide who knows what about you and how that information is used in AI systems.

The GDPR and the EU AI Act complement each other: the GDPR regulates the handling of DATA, the AI Act regulates the AI SYSTEM itself. Both apply simultaneously to an AI product in the EU — neither alone is sufficient. At one point they interlock directly: to detect and correct biases in training data, the AI Act exceptionally permits processing specially protected data such as health data. Originally this applied only to providers of high-risk systems (Art. 10(5)); since the AI Omnibus (Regulation (EU) 2026/1744, in force since July 2026), it also applies to other AI systems and models, provided the bias could endanger health, safety or fundamental rights or lead to prohibited discrimination (Art. 4a AI Act). The conditions are strict: only where strictly necessary and where synthetic or anonymized data will not do, with pseudonymization and access controls, and with deletion once the bias has been corrected.

Interactive: Do I Need a Data Processing Agreement?

Answer the questions and find out which GDPR measures are relevant for your AI project.

Note

Key Takeaways

  1. The GDPR does not ban AI — it bans uncontrolled processing of personal data. For training, legitimate interest is the most important legal basis: the EDPB has recognized it under conditions since December 2024, and an explicit rule in the GDPR is under negotiation (as of November 2026).
  2. The "right to be forgotten" collides with how neural networks work: information is distributed across billions of weights and cannot be surgically removed. Machine Unlearning is an active but unsolved research field.
  3. Citizens have effective tools: access to their data and, for automated decisions, an understandable explanation (Art. 15), objection to training based on legitimate interest (Art. 21), and protection against solely automated decisions with significant effects (Art. 22).

Check

Quiz: GDPR and AI

Not completed

What does the GDPR principle of data minimization require?

Select one answer

Answer Key: 1) B · 2) C · 3) B · 4) D

Check

Comprehension Check

Can you explain this in your own words? Tick what you've got.

0 of 3 done