Scope note first: this covers the engineering architecture that makes privacy compliance achievable. Whether a specific processing activity is lawful, what your basis is, and whether a DPIA is required are legal determinations for your counsel or DPO. No vendor can make them for you, and any who claim to should be treated cautiously.
Written for teams building LLM applications that will process European personal data — the architectural decisions that make compliance possible rather than painful.
Where personal data ends up in an AI system
More places than teams expect, which is the root of most problems:
| Location | Often overlooked? |
|---|---|
| Source documents | No |
| Chunked text in the vector index | Frequently |
| Embeddings themselves | Almost always |
| Prompts sent to the model provider | Sometimes |
| Cached responses | Frequently |
| Application and trace logs | Very frequently |
| Evaluation sets built from real queries | Almost always |
The last row catches teams out repeatedly. Evaluation sets are built from real user queries because synthetic ones are less useful — which means they contain personal data, live outside your main system, and are frequently shared with vendors. Treat them as a data store with the same controls as any other.
The architectural principles
1. Minimise at ingestion
The cheapest compliance strategy is not collecting what you do not need. Before indexing a corpus, ask whether personal identifiers need to be in it at all — many knowledge bases work perfectly with names redacted.
2. Keep an authoritative source
Every derived copy — chunk, embedding, cache — should be traceable back to a source record. This is what makes deletion tractable: delete the source, and a documented process removes every derivative.
3. Design deletion before you need it
- Can you locate every chunk derived from a given source document?
- Can you remove those vectors from the index without a full rebuild?
- Do cached responses referencing that content get invalidated?
- Are logs containing the data within retention limits, or purgeable?
If you cannot answer those four questions, you cannot honour a deletion request — and discovering that when the request arrives is the wrong time.
4. Control what leaves your environment
- Send the minimum context necessary, not everything retrieved.
- Redact identifiers where the task does not require them.
- Use provider tiers with zero retention and no training use — and confirm in writing.
- Know and be able to state the processing location.
5. Log deliberately
Observability requires logging prompts and responses, which means logs contain whatever those contained. Decide explicitly what is logged in full, what is redacted, retention period, and who has access.
Vendor questions worth asking in writing
- Which tier are we on, and what is the data retention policy on it?
- Is our content used for model training or improvement?
- Where is processing performed geographically? Can a region be specified?
- What is logged on your side, and for how long?
- What sub-processors are involved?
- What is the incident notification process and timeline?
Get these answered in the contract or a data processing agreement, not in a sales call.
Practical design patterns
| Requirement | Architectural response |
|---|---|
| Data minimisation | Redact at ingestion; send minimal context |
| Purpose limitation | Separate indexes per purpose, not one shared corpus |
| Right to erasure | Source-to-derivative traceability; index deletion capability |
| Right of access | Ability to report what is held about a person |
| Storage limitation | Enforced retention on logs and caches |
| Data residency | Regional provider endpoints; regional infrastructure |
| Accountability | Audit logs of processing activity |
The self-hosting question
Where personal data genuinely cannot leave your environment, self-hosted inference removes the transfer question entirely. That is a legitimate driver — but it brings substantial operational burden, and it does not remove your other obligations. Minimisation, retention and deletion apply just as much to a model you run yourself.
A build sequence that stays compliant
- Map the data — what personal data enters the system and why.
- Involve legal early — lawful basis, DPIA requirement, vendor terms review.
- Minimise at ingestion before building anything on the corpus.
- Build deletion capability alongside indexing, not afterwards.
- Configure retention on logs, caches and evaluation sets.
- Document the processing — what happens to data, where, and under whose terms.
- Test deletion end to end before go-live.
Building AI that will process European personal data? Tell us what data is involved — and involve your DPO early, because it shapes the architecture. See our AI agent service, secure data access, and self-hosted vs API.