How Netflix Democratized ML Building a Universal Model Lifecycle Graph

The Fragmentation Crisis in ML at Scale

As Netflix's machine learning footprint expanded from personalization to studio workflows, payments, ads, and beyond, a silent killer emerged: metadata fragmentation. Each domain operated its own tech stack, its own model registry, its own pipeline orchestrator, and its own experimentation platform. ML practitioners had to context-switch across half a dozen tools just to answer a simple question like "Which A/B tests are using this model?"

This is not a Netflix-specific problem. Any organization scaling ML across multiple business units eventually hits this wall. The symptoms are universal:

• Black box models: No centralized way to discover what models exist, what features they use, or who owns them.
• Duplicated effort: Teams unknowingly build similar embeddings or features because they can't find existing work.
• Impact blindness: Changing a feature or retiring a pipeline is risky because downstream dependencies are invisible.
• Onboarding friction: New team members spend weeks learning tribal knowledge about which tools to use and where to find things.

Netflix's answer was the Metadata Service (MDS) — a system that ingests events from every ML tool, normalizes them into a unified entity model, and builds a Model Lifecycle Graph that enables discovery, lineage, and impact analysis in a single query.

Reference: Netflix Tech Blog

Architecture Deep Dive: From Events to a Connected Graph

MDS follows a multi-stage pipeline that transforms raw system events into a rich, traversable graph. Let's walk through each stage with a concrete example: connecting a ranking model to its A/B tests.

Stage 1: Event Ingestion

Source systems (model registry, pipeline orchestrator, feature store, experimentation platform) emit thin events via Kafka or AWS SNS/SQS. These events are minimal — just an identifier and an event type. The key insight: events are notifications of change, not logs of state.

{
  "event_type": "model_instance_created",
  "instance_id": "ranking-model-v5-20250101"
}

Stage 2: Entity Enrichment

When MDS receives an event, it doesn't trust the event payload. Instead, it calls the source system's API to fetch the complete, current state. This "hydration" pattern makes the system robust to out-of-order or dropped events — the next event always corrects the state.

# Pseudo-code for enrichment worker
def handle_model_created_event(event):
    # Fetch latest state from source of truth
    model_data = call_model_registry_api(
        f"/api/v1/instances/{event['instance_id']}"
    )
    # Transform to normalized entity
    entity = normalize_entity(model_data, event)
    # Store in graph database
    datomic_client.write(entity)
    # Index for search
    elasticsearch_client.index(entity)

Stage 3: Normalization

Raw events have heterogeneous schemas. MDS normalizes them into a unified model using AIP URIs (e.g., aip://model/registry/ranking-model-v5-20250101). This creates a consistent vocabulary across all domains.

Stage 4: Storage and Indexing

Normalized entities are written to Datomic (for relationship-heavy graph traversals) and indexed in Elasticsearch (for fast full-text search). This dual-storage approach is critical: users typically start with a search query and then navigate the graph.

Stage 5: Knowledge Enrichment

This is where the magic happens. Background jobs discover relationships that no single source system knows about. For our ranking model example:

1. The model references a pipeline_run_id.
2. The enrichment job fetches pipeline metadata and discovers it was executed for A/B test cell #2 of test "Ranking Model v5 vs v4".
3. MDS materializes this transitive relationship: Model Instance → Pipeline Run → A/B Test Cell → A/B Test.

Now, a single GraphQL query answers what previously required four separate tools:

query {
  model(id: "aip://model/registry/ranking-model-v5-20250101") {
    name
    owners { name }
    associatedAbTests {
      name
      cells { number name }
    }
  }
}

Limitations and Caveats

• Enrichment latency: Relationships are discovered asynchronously (minutes, not seconds). Practitioners need to be aware that newly created entities may not immediately appear in the graph.
• Source system reliability: If a source system fails to emit events or returns stale data, MDS can propagate that staleness. The system is only as good as its inputs.
• Graph completeness: Not all relationships are captured. Implicit connections (e.g., two models serving similar purposes based on shared features) require advanced inference that is still in early exploration.

Why This Matters Beyond Netflix

The Model Lifecycle Graph pattern is becoming a best practice for any organization scaling ML. The core ideas are transferable:

1. Treat metadata as a first-class citizen. Don't let each tool silo its own metadata. Invest in a central ingestion and normalization layer.
2. Use URIs for universal addressing. A consistent naming scheme (e.g., aip://domain/provider/entity-id) makes cross-system references trivial.
3. Embrace asynchronous enrichment. You don't need real-time relationships. Background jobs that walk the graph and materialize edges are good enough for most use cases.
4. Design for discovery first. Most ML practitioners don't know what they don't know. A searchable, browsable graph surfaces hidden assets and reduces duplicated effort.

For teams just starting this journey, a pragmatic first step is to build a simple metadata catalog that indexes models, features, and pipelines from two or three key systems. Start with the question: "What are the top 5 things my ML practitioners can't find today?" and solve those first.

Next Steps

• Explore open-source alternatives like Amundsen or DataHub that provide similar metadata cataloging capabilities.
• Read about Python Typing in 2025: 86% Adoption & The Challenges That Remain for insights on how type safety can improve ML pipeline reliability.
• Check out Azure's AI Datacenters Built for NVIDIA's Rubin Platform to understand the infrastructure trends powering next-gen ML workloads.

Conclusion: The Graph is the New Source of Truth

Netflix's MDS demonstrates that the hardest part of scaling ML isn't the algorithms — it's the organizational and infrastructural plumbing. By building a unified Model Lifecycle Graph, Netflix turned a fragmented landscape of black-box models into a discoverable, queryable, and reusable asset ecosystem.

The key takeaway for ML engineers and platform teams: invest in metadata infrastructure early. The cost of retrofitting discovery and lineage after fragmentation sets in is far higher than building a solid foundation from day one. Start small, connect your most critical systems, and let the graph grow organically. Your future self — and your new team members — will thank you.