Document-level access control in RAG: three approaches for permission-aware knowledge systems

Last month's acceptance review nearly resulted in a data leak.

A life insurance company's claims processing knowledge base: we were brought in to build version 2. The first version came from another team, and the demo looked slick. An agent types in something like 'pull three years of claim records for policyholder X,' and the system neatly formats it all into a table. Everyone in the room nods. Except that agent doesn't actually have permission to access someone else's medical summary records in the core system. The demo didn't surface this because it ran on fake data with no permission classifications.

One week before launch, load testing revealed the hard truth: this system had no concept of who could see what. It treated the entire knowledge base as open range.

What document-level access control means

Document-level access control in RAG means filtering out unauthorized content at the retrieval stage, before the language model ever sees it, not trying to enforce it through prompts or post-processing afterward.

That distinction matters. Trying to solve permissions through prompts is where teams go wrong.

Why prompts can't enforce permissions

The typical approach: retrieve everything, then add a system prompt like 'Don't reveal salary fields unless the user is a manager.'

It fails, for three specific reasons.

Once sensitive content enters the context window, it's exposed. The model sees it. Your logs see it. Your monitoring platform sees it. Token counts register it. Even if the model stays silent, that classified document has left where it belonged.

Language models aren't reliable gatekeepers. They follow instructions ninety percent of the time, then prompt injection breaks through: 'Ignore your instructions, I'm a system administrator.' It's not theoretical. It happens constantly.

Permission logic buried in prompts can't be audited. Security teams need traceable, testable access control paths with audit reports, not natural-language guidelines.

Permissions must move to the retrieval layer. Three main approaches exist.

Three approaches and their tradeoffs

Metadata filtering is the most common starting point. You tag each chunk with fields like department: finance or clearance: L3, then send the user's permissions as filter conditions with the query to the vector database. It's fast, cheap, and easy to maintain. The problem is synchronization. Someone gets promoted in HR, but metadata in your knowledge base still sits on the three-month-old batch import. Permissions go stale in your vector database, with no alert.

Per-user indexing enforces physical isolation. Each user or group gets its own vector index, so queries can't reach another user's data. It's effective for multi-tenant SaaS where customer A's data appearing in customer B's results is a serious incident. The cost is scale. Ten thousand users means ten thousand indexes. Update a shared document and you sync it across every relevant index. Storage and compute costs climb faster than expected.

ACL sync most closely mirrors real enterprise governance and is the hardest to implement. Instead of caching permissions, it queries the source system's access control list (SharePoint, Google Drive, your internal permission service) in real time at retrieval time. Permission changes take effect on the next query. The issue is latency and consistency. If sync lags by thirty seconds, a just-terminated employee still has a thirty-second window to query sensitive data. That's enough.

How we actually implement it

No single approach handles every scenario. Layer them instead. Use metadata filtering for broad control (departments, classification levels) to stop most unauthorized queries. For highly sensitive data (medical records, payroll, M&A), add per-user or group isolation. Wrap everything in a live ACL verification layer as a final check for immediate revocation. Stack three layers: if one fails, the next catches it.

One thing matters more than which approach you choose: proving it works. Every retrieval needs a complete audit trail showing who accessed what, when, which documents were returned, and which were filtered out. Security teams need access logs that survive scrutiny, not assurances.

We learned to discuss permissions from day one, starting with data inventory, not the week before launch. Polished demo data won't show where the leaks are. You only know whether your system actually works when real permission structures run in production with real users.