Automated Data Classification Architecture for DLP (2026)

Q: Why does data loss prevention fail without an automated classification layer?

Because DLP is a policy engine that needs to know what a piece of data is before it can decide what may happen to it, and without classification it has to infer that at the moment of egress from the bytes passing through a proxy, a mail gateway or an endpoint agent. Those bytes are a copy that has been joined, renamed, truncated, exported and reformatted since the data was created, so pattern matching at that point has poor precision and unknown recall. Engineers learn to route around rules that block the wrong things, and the program is remembered for the spreadsheet it stopped rather than the breach it prevented. A classification layer that assigns a label at ingest and propagates it to every copy lets the DLP rule read a label instead of guessing, which is the difference between a rule that says "block anything that looks like a card number" and one that says "block exports from assets labelled Restricted".

Q: What is the difference between data classification and data discovery?

Discovery finds assets: the buckets, tables, drives, SaaS exports and vector stores that exist, including the ones nobody registered. Classification assigns each discovered asset and column a sensitivity label. They are distinct problems that fail independently. A program can classify 98 percent of the assets it scans while the scanned estate is 40 percent of what discovery would have found, which is why coverage must always be reported against the discovered estate rather than the scanned one. Unmanaged object storage, personal drives and ad-hoc exports are discovery failures that no classifier can fix. In the architecture, discovery feeds the catalog, the classification engine runs over what the catalog knows about, and the propagation service reports the assets it could not reach so that discovery and rescanning are targeted at the gaps.

Q: How do you decide which classification method to use for which data?

By cost and by what each method is good at, arranged as tiers so that the expensive method only runs on what the cheap ones could not resolve. Pattern and checksum detection (Purview sensitive information types, Macie managed data identifiers, Google Cloud infoTypes) is cheap and precise on formatted identifiers such as card numbers, national IDs and API keys, so it runs everywhere first. Exact data match against a fingerprinted reference set from a system of record catches your own customers' values in free text and runs second on crown-jewel identifiers. Trained classifiers handle categories rather than identifiers, such as contracts, medical notes and source code, in unstructured stores. LLM-assisted semantic inference, which reads a column's name, samples, comments and schema neighbours, has the best recall on ambiguous columns and the highest cost per column, so it runs only on what remains, with a confidence threshold and a steward queue behind it.

Q: How does label propagation along data lineage work in practice?

The propagation service walks the lineage graph the catalog already holds and writes inherited labels onto derived assets. A derived table or view inherits the highest label among the source columns that flow into it, unless the transformation is registered as declassifying, for example a k-anonymised aggregate, a keyed hash with the key held elsewhere, or a tokenised value issued by a vault. Because inheritance costs no scanning, every derived asset gets a label the moment it is created, and rescans are reserved for what lineage cannot see. The service also emits an inventory of assets with no lineage edge, such as manual exports, notebook writes to object storage and pulls by external tools, and that inventory becomes the targeted rescan schedule. Where lineage capture does not exist, propagation degrades to scheduled full rescans, which is a sign that lineage is the first investment to make.

Q: What does LLM-assisted data classification cost, and how do you keep it under control?

It costs tokens per column per pass, so the two levers are how many columns reach the LLM tier and how often. In a worked example, an estate of one million columns where pattern matching resolves 60 percent and exact match plus trained classifiers resolve half of the remainder leaves 200,000 columns for LLM inference. At roughly 1,500 tokens of context per column that is about 300 million tokens for a full pass, which at an illustrative small-model price of $0.20 per million input tokens is around $60, and a monthly rescan of only the lineage-gap inventory is a fraction of that. Running the same inference on all one million columns weekly instead multiplies the spend by five and the pass count by four for no improvement on the 800,000 columns the cheaper tiers had already resolved. The cost of the LLM tier is therefore a measure of how well the cheap tiers and lineage propagation did their jobs.

Q: Should uncertain classifications default to a permissive or a restrictive label?

Restrictive. A low-confidence column labelled "internal" so that nothing breaks is a data breach waiting for the first export, and the one wrong "internal" on a health record is the incident that ends the program. The workable policy is to apply labels above a confidence threshold directly and audit them by sampling, and to apply labels below the threshold provisionally at the more restrictive level, reviewing them only when someone requests an access that the provisional label blocks. That converts an unbounded steward queue, which a program routing 5 percent of a million columns to review would fill with 50,000 items nobody clears, into a queue sized by actual demand for the data. Provisional restrictive labels also age well: data nobody asks for stays protected without anyone spending a minute on it.

Q: How does data classification apply to RAG indexes, copilots and fine-tuning data?

A retrieval index, an enterprise search index, a fine-tuning corpus and a vector store are all copies of the underlying data for classification purposes, and they are the copies most programs forget. The AI adapter in the architecture translates the canonical label into three controls: exclusion of labelled sources from an index entirely, a per-chunk metadata filter so that retrieval respects the requesting user's entitlement to the label, and a gate on training-set assembly so that Restricted data cannot enter a fine-tuning run without a recorded decision. Without that adapter the copilot reads everything the catalog never labelled and every user with the copilot effectively has every document the index saw. Products marketed in 2026 as DSPM for AI, including Microsoft Purview's, are vendors addressing this gap; the label they act on still has to come from a classification layer that propagated it to the index.

Q: When is building a classification layer the wrong first project?

When the sensitive-data estate fits on one page. An organisation whose regulated data lives in three systems it can enumerate by hand is better served by enforcing controls store by store than by standing up discovery, a tiered engine, a label store and propagation. The trigger for building the layer is the month that list stops being true, which in most companies is when the second warehouse, the first vector store or the first enterprise copilot arrives and the number of copies of any given customer record stops being knowable. It is also the wrong project, or at least the wrong first project, in an estate with no lineage capture, because the backbone of the design is inheritance and there is nothing to inherit along; there the first investment is lineage, after which classification becomes a propagation problem rather than a perpetual rescan.

Satyam Kumar

返回博客

cyber-security-patterns

Automated Data Classification: The Prerequisite Every DLP Program Skips, and Why Labels Must Propagate or Die (2026)

By Satyam KumarSeptember 27, 202617 min read

automated data classification data classification architecture sensitivity labels label propagation lineage data loss prevention prerequisite dspm dspm for ai microsoft purview classification amazon macie google sensitive data protection snowflake data classification databricks unity catalog data classification trainable classifiers llm-assisted data classification data catalog governance exact data match

Automated Data Classification: The Prerequisite Every DLP Program Skips, and Why Labels Must Propagate or Die (2026)

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

Automated Data Classification: The Prerequisite Every DLP Program Skips, and Why Labels Must Propagate or Die (2026)

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment