Data Protection and AI: When the GDPR Meets Neural Networks
The GDPR and language models — a relationship that needs clarification.
Italy halted ChatGPT in March 2023 — not because AI is illegal in Europe, but because the Italian data protection authority accused OpenAI of having no valid legal basis for processing the data of millions of people. The GDPR does not forbid AI. It forbids careless data handling.
Two GDPR principles — data minimization and purpose limitation — together with the right to be forgotten create genuine technical conflicts with how modern AI systems are built. This article unpacks the three central flashpoints and explains what rights you have as a citizen.
GDPR: A Turning Point for Data Protection with AI Impact
On May 25, 2018, the EU General Data Protection Regulation (GDPR) became applicable - a turning point for AI and data protection worldwide. As the 'mother of all data protection laws,' it replaced the outdated Directive 95/46/EC from 1995, a relic of the internet's early days. GDPR made Privacy by Design mandatory: data protection must be built into AI systems from the ground up. Its global reach was enormous - even US tech giants must comply with EU standards when processing European data. For AI, this posed a fundamental challenge: how do you explain 'black box' algorithms when GDPR demands transparency? Observers saw an incentive to develop AI with a leaner data footprint, and methods such as transfer learning gained in importance. GDPR inspired data protection laws around the world, from California to Brazil. The regulation laid the groundwork for the EU AI Act in 2024 - the step from data protection to AI regulation was a logical one.
Data Minimization vs. Scaling Laws
Definition
Data Minimization (Art. 5(1)(c) GDPR)
Analogy
Definition
The analogy suggests the only solution is to stop trawling. In reality, technical measures like anonymization, synthetic data, or differential privacy can partially reconcile the two approaches.
More data means a better model. Large language models need billions of texts — inevitably containing personal data.
Only the minimum necessary data may be processed. Every processing operation requires a legal basis and a defined purpose.
Purpose limitation makes this harder: a fitness app collects your data to analyze your workouts — not to train an AI model. Data may only be reused for a new purpose if that purpose is compatible with the original one (Art. 6(4) GDPR) or if a separate legal basis applies. For health data, Art. 9 adds a further hurdle, and often only explicit consent remains. Pseudonymization helps, but it changes nothing for the provider itself: whoever can link the data back to the individuals is still processing personal data. Only for a recipient who realistically cannot re-identify the individuals can the data lose its personal character, according to a 2025 ruling of the Court of Justice of the European Union.
So which legal basis covers training? Consent is hardly practical for billions of texts from the web. The research privilege (Art. 89 GDPR) eases reuse for scientific purposes, but it is not a legal basis in itself. In practice, it usually comes down to legitimate interest (Art. 6(1)(f)). For a long time it was unclear whether that holds for AI training — by now the line is clearer. In its Opinion 28/2024 of December 2024, the European Data Protection Board (EDPB) considers it possible if a three-step test is passed case by case: the provider pursues a legitimate interest, the processing is necessary for it, and the interests of the data subjects do not override it — which also depends on whether they could reasonably expect such use. Along these lines, the Higher Regional Court of Cologne in May 2025 rejected an urgent application by consumer advocates against Meta training on public Facebook and Instagram posts. In July 2026, the EDPB also put guidelines on web scraping for generative AI out for public consultation, which spell out this balancing test further with examples. The European Commission additionally wants to anchor legitimate interest for the development and operation of AI explicitly in the GDPR (proposal for a "Digital Omnibus" on data and the GDPR of November 2025, new Art. 88c — not to be confused with the AI Omnibus proposed at the same time, which has already amended the AI Act), combined with an unconditional right of data subjects to object. The dispute is over whether this turns into a free pass for providers and how strong the right to object should be. As of November 2026, the proposal has not been adopted: the member states agreed on their negotiating position in October 2026 and dropped the unconditional right to object, leaving the general right to object under Art. 21. The European Parliament has not yet settled on its position. Until an amendment is adopted and enters into force, the GDPR applies in its current form. How quickly things can escalate without a solid legal basis is shown by the Italy case:
- 1. March 2023: The data protection authority Garante temporarily halts ChatGPT in Italy — among other things, a legal basis for training is said to be missing
- 2. April 2023: OpenAI adds privacy notices, an objection form and an age check — ChatGPT is available again
- 3. December 2024: The Garante imposes a fine of 15 million euros — training without a legal basis defined in advance, insufficient information, an unreported data breach, no age verification
- 4. March 2026: A court in Rome annuls the decision — not on the merits, but because OpenAI had meanwhile set up an establishment in Ireland, making the Irish authority the lead authoritythe GDPR's one-stop shop
Caution
Common Misconception: "The GDPR bans AI in Europe"
The GDPR does not ban AI — it bans uncontrolled processing of personal data. AI is legal under the GDPR when it is developed with a solid legal basis (such as consent or legitimate interest) and safeguards such as anonymization. Italy did not permanently ban ChatGPT — it required compliance.
The Right to Be Forgotten — A Technical Dilemma
Definition
Right to Erasure (Art. 17 GDPR)
Analogy
Definition
There is a crucial distinction: training data (can be deleted from servers) vs. model weights (information distributed, cannot be surgically removed). One is a file problem; the other is a math problem.
What happens when a person demands that ChatGPT "forget" them? OpenAI can activate an output filter (the name no longer appears in responses) and exclude the person from future training data. But the existing model may still contain the information in its weights — it then simply no longer expresses it. The person may be hidden, not forgotten.
Machine Unlearning is an active research field: how do you make a model specifically forget certain data? Bourtoule et al. (2021) showed an exact approach with their SISA method: the training data is split into shards from the outset, a sub-model is trained per shard, and after a deletion request only the affected shard needs retraining. However, this has to be planned before training and is considered hardly practical for large language models. For models that are already trained, only approximate methods exist, and surveys from 2025 and 2026 reach the same conclusion: these methods mostly just suppress the knowledge — targeted attacks or a little further training can often bring it back. As of November 2026, there is no reliable, verifiable method. The core question remains: how do you prove that a model has really forgotten?
Carlini et al. (2021) showed that hundreds of verbatim text sequences from the training data can be extracted from GPT-2, including names, phone numbers, and email addresses; Nasr, Carlini et al. (2023) managed this on a much larger scale, even with ChatGPT — despite safety training. This underscores: filters hide information, they do not delete it.
Caution
Common Misconception: "Output filters solve the deletion problem"
Output filters prevent the model from displaying information. But the data may remain encoded in the weights and can be extracted under certain conditions. Whether model weights constitute personal data at all was disputed among supervisory authorities — the Hamburg Commissioner for Data Protection and Freedom of Information denied it in a 2024 discussion paper. In December 2024, the EDPB set the line: case-by-case assessment. A model is only anonymous if it is very unlikely that individuals can be identified from it and that their data can be extracted through targeted queries. Filters are a pragmatic workaround, not a legal solution under the GDPR.
A second, related problem: language models sometimes invent false claims about real people. The GDPR requires data to be accurate (Art. 5(1)(d)) and gives you a right to rectification (Art. 16). But a model is about as hard to correct in a targeted way as it is to make it forget. The privacy organization noyb has therefore filed complaints against OpenAI, in Austria in 2024 and in Norway in 2025: there, ChatGPT had falsely described a man as the convicted murderer of his children. No decision by the authorities is known so far (as of November 2026).
Your Rights, Their Obligations
How seriously the Court of Justice of the European Union takes these rights is shown by two rulings on creditworthiness. In the SCHUFA case (December 2023), it held that an automatically calculated score is itself a decision within the meaning of Art. 22 if the bank relies on it decisively — the credit agency cannot hide behind the fact that, formally, only the bank decides. In the Dun & Bradstreet Austria case (February 2025), it clarified the right of access: anyone assessed automatically is entitled to an understandable explanation of the procedure and principles actually applied. Merely handing over a formula or the algorithm is not enough, and trade secrets do not justify a blanket refusal — in a dispute, the supervisory authority or the court weighs the interests.
Think of a building permit system: before you construct a building (deploy an AI system), you need two permits. One from the building authority (EU AI Act — is the building safe?) and one from the environmental agency (GDPR — are the neighbors' rights respected?). Both must approve before construction begins. Unlike in construction, though, there is usually no authority issuing a permit in advance — the company must check and prove on its own that it complies with both sets of rules.
For a German company deploying a customer service chatbot, this can mean in practice:
- Conduct a Data Protection Impact Assessment (DPIA) before launch (Art. 35 GDPR) — AI customer support involving personal data is on the mandatory list of the German supervisory authorities
- Define a legal basis for the chat logs: for handling the request, contract performance or legitimate interest is usually sufficient; if the logs are to serve further purposes, such as training the company's own models, a separate legal basis is needed — often consent. Inform customers in any case (Art. 13)
- Sign a data processing agreement with the LLM provider (Art. 28 GDPR) — more on this in the decision tree below
- Remove or pseudonymize personal details where possible before sending them to the LLM
- Set a retention period and delete the chat logs afterwards, for example after 30 days
- Label the chatbot as AI — required by Art. 50(1) EU AI Act since 2 August 2026, unless this is obvious anyway
Caution
Common Misconception: "If you have nothing to hide, you have nothing to fear"
Data protection does not primarily protect secrets — it protects the fundamental right to informational self-determination. It is about your power to decide who knows what about you and how that information is used in AI systems.
The GDPR and the EU AI Act complement each other: the GDPR regulates the handling of DATA, the AI Act regulates the AI SYSTEM itself. Both apply simultaneously to an AI product in the EU — neither alone is sufficient. At one point they interlock directly: to detect and correct biases in training data, the AI Act exceptionally permits processing specially protected data such as health data. Originally this applied only to providers of high-risk systems (Art. 10(5)); since the AI Omnibus (Regulation (EU) 2026/1744, in force since July 2026), it also applies to other AI systems and models, provided the bias could endanger health, safety or fundamental rights or lead to prohibited discrimination (Art. 4a AI Act). The conditions are strict: only where strictly necessary and where synthetic or anonymized data will not do, with pseudonymization and access controls, and with deletion once the bias has been corrected.
Interactive: Do I Need a Data Processing Agreement?
Answer the questions and find out which GDPR measures are relevant for your AI project.
Note
Key Takeaways
- The GDPR does not ban AI — it bans uncontrolled processing of personal data. For training, legitimate interest is the most important legal basis: the EDPB has recognized it under conditions since December 2024, and an explicit rule in the GDPR is under negotiation (as of November 2026).
- The "right to be forgotten" collides with how neural networks work: information is distributed across billions of weights and cannot be surgically removed. Machine Unlearning is an active but unsolved research field.
- Citizens have effective tools: access to their data and, for automated decisions, an understandable explanation (Art. 15), objection to training based on legitimate interest (Art. 21), and protection against solely automated decisions with significant effects (Art. 22).
Check
Quiz: GDPR and AI
What does the GDPR principle of data minimization require?
Select one answer
1. What does the GDPR principle of data minimization require?
2. Italy's data protection authority halted ChatGPT in March 2023. A company wants to launch an AI chatbot for its customer service in Germany. Which measure is NOT required by the GDPR?
3. A user invokes their "right to be forgotten" (Art. 17 GDPR) against an LLM provider. The provider activates an output filter that suppresses the user's name in responses. Why does this NOT fully satisfy the GDPR requirement?
4. A bank uses an AI system that rejects loan applications without human review. A customer's application is rejected. Which GDPR article targets the fully automated decision itself?
Check
Comprehension Check
Can you explain this in your own words? Tick what you've got.
0 of 3 done