Handling PII and sensitive data in enterprise RAG: de-identification, masking, and compliance implementation

Late last year, we conducted acceptance testing for an internal knowledge assistant at a regional bank. An engineer asked for last quarter's credit review templates, and the system returned a customer's complete ID number and account balance from the indexed internal documents.

No attack occurred. The system was working as designed: retrieve information and respond.

The issue starts at ingestion. Most enterprises load documents into a vector database and ask questions, forgetting how much sensitive information those documents contain. Blocking the model's response doesn't solve the problem. The solution is making decisions at ingestion time: which fields should be de-identified, which should be masked, and which shouldn't be indexed at all. This outlines the governance process we deployed.

PII and sensitive data handling in enterprise RAG: the four layers

The process consists of four sequential steps in fixed order: Detection → De-identification → Masking → Audit.

Detection is the first layer. You begin by understanding what's in the corpus. Taiwan's Personal Data Protection Act defines personal data broadly in Article 2: names, ID numbers, medical records, financial information, social activities, and even indirect descriptions that identify a specific person. The implementation combines rule-based pattern matching with an NER model. Rules identify structured items (ID numbers by check digit, company registration numbers, credit card numbers using Luhn validation). The NER model identifies context-dependent cases like "Mr. Wang's court case at Northern Hospital last month," which surface only through surrounding text. Run detection before chunking. If one person's information splits across two chunks, the rules will miss it.

De-identification is the second layer. The decision at this stage is whether data should be recoverable. Confusion often begins here.

Financial services should note that pseudonymization remains personal data under Taiwan's Personal Data Protection Act because it's theoretically reversible; key management therefore requires sensitive-data-level security controls. Healthcare operates under the Human Biobank Management Act, which requires data to be de-linked before it exits personal data scope. Generalization and k-anonymity hold up better than simple masking in medical contexts.

Masking is the third layer, and it must be role-aware. A compliance officer sees the full document; customer service staff see only masked content. Mark each chunk's sensitivity level in metadata, then include the user role in the retrieval filter. This allows the vector database to pre-filter results before returning them, rather than retrieving everything and hoping the prompt keeps the model quiet. The model will find ways around that approach.

Audit: what isn't logged didn't happen

The fourth layer is easily skipped but essential for compliance. Article 18 of Taiwan's Personal Data Protection Act requires both public and private entities to implement "appropriate security safeguards." Taiwan's Financial Supervisory Commission explicitly requires that access trails be auditable.

Each retrieval requires a log entry: who accessed it, when, what they queried, which chunks returned, and whether de-identification triggered. The audit log itself is sensitive, so it needs separate storage, encryption, and a retention policy. A common mistake: logs get written carefully but stored at the same permission level as the main database, consolidating all sensitive queries into a larger target. The better approach is moving audit trails to read-only isolated storage that only the audit role can access.

The difficult part after launch isn't selecting a de-identification algorithm; it's whether someone will work through each data source to identify sensitive fields and decide what each user role should see. No off-the-shelf tool handles this work. When we implement RAG for customers, the entire first week goes to working with compliance and security teams, not writing pipeline code. The goal is completing the "field × role × technique" matrix. A system that works in a demo means nothing. What matters is what the system actually does after it ships.