A colleague pastes a customer email into a free AI chatbot to draft a polite reply. Quick, convenient, and probably well-intentioned. But that email contains a name, an email address and a complaint about an invoice. Personal data has just been shared with an external party — and that means the GDPR applies.
This article explains what personal data actually is, why consumer AI tools are a problem here, what a lawful basis means, and above all: what to do and not do in practice. No scare story, just the basics everyone who works with AI should know.
What counts as personal data?
The GDPR defines personal data broadly: any information relating to an identified or identifiable natural person. That covers far more than a name or ID number. Think of:
- Direct identifiers: name, email address, phone number, home address, photo, employee number.
- Indirect identifiers: a combination of role, department and age that points to one person. “The team lead at the Zwolle branch who just came back from parental leave” is personal data, even without a name attached.
- Special categories: data about health, religion, ethnicity, political views or trade union membership. These carry much stricter rules; processing them is prohibited in principle, unless a specific exception applies.
The rule of thumb: could someone use this information (possibly combined with other information) to single out a specific person? Then it is personal data. When in doubt, treat it as if it is.
Why is pasting into a consumer chatbot a problem?
Free and consumer versions of AI tools are built for private use, not for processing business data. Several concrete problems arise:
1. You do not know where the data goes
Whatever you enter leaves your organisation and ends up on the provider’s servers. Depending on the provider and your settings, that input may be stored, reviewed by the provider’s staff for quality purposes, or used to improve models. In many consumer versions, model training on your input is switched on by default and you have to opt out yourself — if that option exists at all.
2. There is no data processing agreement
When your organisation has personal data processed by an external party, the GDPR requires a data processing agreement (Article 28 GDPR). It sets out what the processor may and may not do with the data. When an employee signs into a free chatbot on their own initiative, no such agreement exists. Business and enterprise versions of AI tools often do provide one — which is exactly why organisations pay for them.
3. You cannot get the data back
Data subjects have rights under the GDPR: access, rectification, erasure. Once customer data has ended up in an external AI system your organisation has no contract with, honouring those rights becomes practically impossible. You do not even know for certain what was stored.
4. Reporting a data breach is mandatory
If personal data ends up somewhere it should not be, that can constitute a data breach. Depending on the risk, your organisation must report it to the supervisory authority — in the Netherlands the Autoriteit Persoonsgegevens — and in some cases inform the people affected. “A colleague accidentally pasted it into a chatbot” does not exempt anyone from that duty.
Lawful basis: the question that gets skipped
The GDPR says you may only process personal data if you have a lawful basis. There are six, including consent, performance of a contract, a legal obligation and legitimate interest. No basis, no processing — full stop.
For AI use, this means: if your organisation collected customer data to send invoices, that does not automatically give it a basis to run the same data through an AI tool for an entirely different purpose. This is called purpose limitation. It is a question for your privacy officer or data protection officer (DPO), not something an individual employee decides on the fly. What you can do as an employee: ask the question before you enter anything.
The core question in one sentence: would you put this information in an email to a random outsider? No? Then it does not belong in an AI tool your organisation has no agreement with either.
Practical do’s and don’ts
Do
- Anonymise before you paste. Replace names with “customer A”, strip email addresses, phone numbers and address details. Note: anonymising is more than removing the name — check whether the person is still identifiable from context.
- Use the tools your organisation has approved. Business AI environments typically come with agreements on data storage and training that consumer versions lack.
- Check your settings. If you do use a personal account for non-sensitive tasks, switch off the option to use your conversations for model training where possible.
- Ask when in doubt. Your privacy contact or DPO would much rather answer ten cautious questions than report one data breach.
- Report mistakes quickly. Fast reporting is the difference between a manageable incident and a serious problem. A healthy organisation does not punish honesty.
Don’t
- Paste complete customer emails, CVs, HR files, meeting notes about individuals or student records into a consumer chatbot.
- Enter health information, financial details or other sensitive data — not even “just this once” and not even partially anonymised.
- Assume that “everyone does it” means it is allowed.
- Connect an AI tool to your work mailbox or documents without your organisation’s approval.
What about the AI Act?
Alongside the GDPR there is the European AI Act (Regulation (EU) 2024/1689), adopted in 2024. It regulates AI systems themselves, where the GDPR regulates personal data. Relevant for the workplace: Article 4 has required organisations, since 2 February 2025, to ensure a sufficient level of AI literacy among staff dealing with AI. Knowing what you can and cannot put into an AI tool is a core part of that. So the two laws overlap on exactly this point: employees who understand what they are actually doing when they use an AI tool.
In summary
- Personal data is any data that can identify someone — directly or indirectly.
- Consumer AI tools offer no data processing agreement and no control; business data does not belong in them.
- Anonymise, use approved tools, and ask your privacy officer when in doubt.
- Something went wrong anyway? Report it fast — that is not failure, that is professionalism.
Want to see how well you and your colleagues judge situations like these? Try the free AI literacy quiz, or see how the AI literacy course covers this topic step by step. For teams, there is a dedicated page for employers.