Why Asset Classification Is the Foundation of Privacy

In the age of AI-native products, data flows through pipelines, becomes features, and feeds models. But before you can enforce retention, access, or anonymization policies, you need to know what your data is. A field named age could be a person's age or a cache TTL. Getting this wrong can trigger false restrictions or leave protection gaps.

Traditional manual review can't keep up with the volume and pace of AI-driven data transformation. That's why Meta's Privacy-Aware Infrastructure (PAI) team developed a hybrid pattern: use LLMs for ambiguity, but distill stable behavior into deterministic, versioned rules for routine enforcement.

This isn't about "LLMs everywhere." It's about building a system that learns from ambiguous signals while keeping production decisions fast, replayable, and auditable.

Privacy-aware infrastructure concept with data classification and access control locks Algorithm Concept Visual

The Pattern: Context, Funnel, Distillation

1. Start with a Contract

Define a small, explicit classification contract. For each asset, return a category, confidence score, decision trace, and version info. Avoid universal taxonomies; each classifier owns one scoped question.

2. Build Context Before Prompting

Most classification failures are context failures, not prompt failures. Create an evidence brief with supporting signals, contradicting signals, and provenance. Mask circular fields to prevent the model from "discovering" the answer.

# Example: Building an evidence brief (simplified)
def build_evidence_brief(asset):
    signals = []
    # Fetch lineage, code references, ownership, semantic annotations
    lineage = get_lineage(asset.id)
    code_refs = resolve_code_references(asset.field_name)
    ownership = get_ownership(asset.id)
    semantic = get_semantic_annotation(asset.field_name)
    
    # Add supporting/contradicting signals with weights
    if lineage.connects_to_user_pipeline:
        signals.append({"type": "supporting", "signal": "lineage", "weight": 0.8})
    if semantic.type == "EMAIL":
        signals.append({"type": "supporting", "signal": "semantic", "weight": 0.9})
    if ownership.team == "infrastructure":
        signals.append({"type": "contradicting", "signal": "ownership", "weight": 0.3})
    
    # Mask circular fields
    masked = mask_existing_label(asset)
    
    return {"signals": signals, "masked_fields": masked}

3. Use a Decision Funnel

Route assets through a deterministic-first funnel. If a versioned rule matches, return the decision. Otherwise, fall back to the LLM. In production, ~85% of traffic resolves via deterministic rules in milliseconds, with the LLM handling the remaining ~15% at higher cost.

# Decision funnel logic
def classify(asset):
    rule = match_rule(asset)
    if rule:
        return rule.decision
    evidence = build_evidence_brief(asset)
    llm_result = call_llm(evidence)
    return llm_result

4. Keep the Learning Loop Safe

Separate reference labels (human-reviewed) from optimization. Use a multi-panel judge with three independent LLM evaluations to validate decisions. Track inter-rater agreement (Cohen's kappa) to detect quality issues.

Data analyst reviewing lineage and context signals for asset classification Coding Session Visual

What We Learned: Pitfalls and Best Practices

  • Context beats prompts: Adding code resolution and lineage fixed a false positive on a age field that was a cache TTL, not a person's age.
  • Determinism means replayability: Versioned rules and prompts allow reproducing decisions for audits.
  • Accuracy alone is misleading: Use balanced metrics like macro F1 and per-class recall to expose rare-class failures.
  • Coverage is not correctness: Track coverage alongside recall to avoid regressions.
  • Self-regulation is architectural: Build a tuning controller that can pause or diagnose when quality drops.

Limitations and Next Steps

This pattern is not a silver bullet. It requires significant investment in context engineering and evaluation. The LLM fallback can be expensive and slow. For teams starting out, focus on building a strong context mesh and a clear contract before optimizing prompts.

Next, explore applying this pattern to other privacy workflows like lineage validation, purpose-boundary checking, and retention policy assignment. The principles also generalize to agent observability and oversight.

AI model reasoning over evidence brief to classify data assets Programming Illustration

Conclusion: Build for the AI-Native Era

Meta's asset classification case study shows that privacy-aware infrastructure can be a driving force for better architecture. By combining LLM reasoning with deterministic rules, you get a system that is secure, scalable, and auditable.

Start small: define a clear contract, build rich context, and let deterministic rules handle the routine. Use LLMs for the ambiguous cases, and always keep human review in the loop for high-risk decisions.

For more foundational knowledge, check out our Python basics guide and Google Cloud Data Commons MCP.

This content was drafted using AI tools based on reliable sources, and has been reviewed by our editorial team before publication. It is not intended to replace professional advice.