Local AI and the GDPR, what it really solves and what it does not

"We run it locally, so we have no data protection problem." You hear that sentence a lot, and it is half right. The half that is true matters enough that local models can be worth it for that reason alone. The half that is not true catches up with you if you do not know about it.

This playbook separates the two. It is not legal advice and does not replace any, it sorts the questions so you know which ones you even have to ask.

1. Understand what local actually removes

The real problem with cloud services is transmission. The moment a piece of personal data goes to a provider, you need a legal basis for it, a settled allocation of roles with the matching contract, and with providers outside the EU an answer to how the transfer is safeguarded. The role question is the one most often answered wrongly: a data processing agreement only fits if the provider genuinely acts as a processor. If it uses the data for its own purposes, it is a controller in its own right or a joint controller, and then something else is needed. That is decided per service and per purpose, not across the board.

If the model runs on your machine, that chain falls away for the inference step. For it there is no recipient, because nothing is received.

What matters, though, is the whole chain and not just the model. A local interface can still call out: web search switched on, a cloud embedding service for document retrieval, a third-party model wired in as a second opinion, telemetry, a cloud backup of the history. In all of those cases your data does leave, even though the model computes locally. Before sensitive documents go in, check once actively: web search off, embeddings local, no third-party model registered, telemetry off, port bound to localhost only, and the backup target demonstrably in your own house rather than in a cloud. Only then does the sentence "nothing leaves the house" hold.

Tip: This is exactly where the value lies for law firms, medical practices, tax advisers and HR departments. Not because the model is better, but because in a properly sealed setup the transmission question never arises.

2. Understand what local does not remove

Everything else still applies. You are the controller in the sense of the regulation, with everything that entails.

You still need a legal basis for the processing itself. An entry in the record of processing activities is due. Data subject rights apply unchanged, so access, rectification, erasure. The principle of data minimisation applies. And if the processing carries high risk, you need a data protection impact assessment, regardless of where the computation happens.

Tip: The most common fallacy is that "on my machine" means "private". A company laptop holding client data is processing, offline too.

3. Establish whether personal data is involved at all

Before you get into the hard questions, check the easy one. Many use cases need no personal data whatsoever. Rewriting a product description, summarizing a technical manual, commenting code, all of that is uncritical.

If you can leave personal data out, leave it out. That is the cheapest solution there is and it costs nothing but a moment of thought.

Tip: Pseudonymisation helps, but it does not make data anonymous. If a name is replaced by an identifier and you keep the mapping table, it remains personal data.

4. Look at where the model itself comes from

The model is a file of weights. It contains no client data, but it comes from somebody, and the licence governs what you may do with it.

Apache 2.0 and MIT are uncomplicated. Other licences carry restrictions, on user numbers or on particular fields of use. For business use that gets checked, not assumed.

Tip: Download models from the official source of the respective family or from your runtime's official library. An anonymously uploaded copy with extra promises in the name is an unnecessary risk.

5. Secure the machine it runs on

When the cloud drops out, you move into the responsibility. The machine with the local model and the processed documents is now the place where the data lives.

Disk encryption, screen lock, access restriction, a backup concept. Those are technical and organisational measures, and with local processing they are not less necessary than before, they are more.

Tip: Check where your interface writes the conversation history. Some drop it unencrypted into a folder in the user directory. That is a collection of your most sensitive inputs in one place nobody thinks about.

6. Do not open the port by accident

Local runtimes listen on your own machine only by default. That is the right setting. Guides nonetheless often tell you to change the binding to all interfaces so you can reach it from your phone.

That makes the model reachable for anyone on the same network, usually without authentication. In an office network, or worse with a port forward in the router, that is wide open.

Tip: If you genuinely need remote access, use an encrypted tunnel rather than an exposed port. The convenience of an open address is not worth it.

7. Remember the logs

Local does not mean traceless. Runtimes write logs, interfaces store histories, some tools park intermediate state in a database.

If personal data ends up there, those are processing operations with a retention period. So it needs deciding how long they stay and who can delete them. Having to answer an access request and then discovering that nobody knows where everything sits is an unpleasant half hour.

Tip: Write down once which three to five places take in data. That list is half the work for the record of processing activities.

8. Keep the EU AI Act separate from the GDPR

These are two different bodies of law, and they get mixed up routinely. The GDPR asks what happens to personal data. The AI Act asks what the system does and what risk comes from it.

The deadlines are staggered, and more importantly Article 50 splits the duties across two different roles. That gets confused constantly.

As a provider of an AI system, paragraph 1 applies to you (disclosing that a person is talking to an AI system) and paragraph 2 (machine-readable marking of generated output). As a deployer using someone else's system, paragraph 4 applies: disclosing deepfakes and marking AI text published on matters of public interest. Paragraph 4 has exceptions, but they are narrower than their reputation: for artistic work the disclosure does not fall away, it is only presented more discreetly, and the exception for text applies only once a human has reviewed the content and somebody carries editorial responsibility. Neither exception touches the provider duties in paragraphs 1 and 2. If you intend to rely on one of them, read the wording rather than this summary.

On timing, 2 August 2026 applies to these transparency duties. For generative systems placed on the market before that date, the deadline for machine-readable marking under paragraph 2 runs until 2 December 2026. The high-risk obligations were pushed back by more than a year by the Digital Omnibus of 8 July 2026; Article 50 was left untouched. Do not quote a concrete high-risk date without checking the Official Journal first, the deferral is recent and secondary sources contradict each other.

In practice that means: work out which role you are in first. Anyone running a local model purely for themselves and publishing nothing is not affected by paragraph 4 at all.

That you run a model yourself changes none of this classification. What counts is the purpose.

Tip: The detail and the delineation are covered at length in DACH legal, EU AI Act and GDPR. This playbook names them only as far as they matter for local setups.

9. State the advantage accurately, not too broadly

When you argue for local processing internally or with a client, phrase it precisely. "The data does not leave our building" is correct and strong. "That makes us GDPR compliant" is wrong, because compliance depends on the whole processing operation and not on where the computation happens.

The second sentence trips you up at the first critical follow-up question. The first one holds.

Tip: Anyone putting a legal statement into a proposal should have it checked by someone who carries liability for it. That is the data protection officer or a lawyer, not the AI and not this playbook.

10. Write the setup down before anyone asks

One page is enough. Which model, on which machine, who has access, which data goes in, what is stored where, for how long, who may delete it.

That single page is the difference between a setup that survives an audit and one where everybody starts reconstructing. Writing it takes half an hour once you have gone through the steps above.

Tip: If several people on the team work with AI, a short rule about what may go in and what may not belongs with it. There is a template in The AI policy for small teams.

What next

If the technical setup is not in place yet, start with Your first local AI model in 30 minutes. The question of the right size is covered by Which local model fits your hardware. And for everyday work with cloud tools, where transmission does happen after all, there is Data protection when working with AI tools.

Source

This playbook is orientation, not legal advice. Deadlines and interpretations change, check them at the primary source.

• Regulation (EU) 2024/1689 in the Official Journal, the authoritative text (Article 50 and everything else): https://eur-lex.europa.eu/eli/reg/2024/1689/oj
• European Commission, enforcement of the AI Act: https://digital-strategy.ec.europa.eu/en/policies/enforcement-ai-act
• Readable rendering of Article 50 (not an official source, for orientation): https://artificialintelligenceact.eu/article/50/
• The deferral of the high-risk deadlines comes from the Digital Omnibus package of summer 2026. Look up the amending act itself via EUR-Lex if you need it; I deliberately link no secondary source for that.
• German Data Protection Conference, guidance on AI and data protection: https://www.datenschutzkonferenz-online.de