Introduction

Managing multi-step business processes across services often requires a central orchestration service, adding operational overhead and a single point of failure. Airbnb's Skipper takes a different approach: it embeds the workflow engine directly into your service, reusing your existing deployment and database. Now open source under Apache 2.0, Skipper offers durable execution without a separate orchestration tier.

Evidence: Airbnb Engineering Blog

Skipper is designed for teams that want workflow execution to stay in-process, avoiding an additional shared dependency for critical services. It's particularly appealing if you already have a robust operational environment and need a workflow engine that adapts to your existing practices.

Server rack representing embedded workflow engine running inside service Algorithm Concept Visual

Core Concepts and Code Example

A workflow is a class. Skipper has two main abstractions: workflows (orchestration logic) and actions (side-effecting operations). Workflows are written in Java or Kotlin.

class PublishListing : Workflow() {
    private val listing = actions()
    @StateField var reviewApproved: Boolean? = null

    @WorkflowMethod
    suspend fun publish(request: PublicationRequest): Boolean {
        if (request.requiresReview) {
            val reviewed = waitUntil(
                { reviewApproved != null },
                Duration.ofDays(1)
            )
            if (!reviewed || reviewApproved != true) return false
        }
        listing.publish(request.listingId, request.requestId)
        listing.notifyHost(request.listingId, request.requestId)
        return true
    }

    @SignalMethod
    fun recordReview(approved: Boolean) {
        reviewApproved = approved
    }
}

The waitUntil call hibernates the workflow without holding a thread, and the recordReview signal can be sent from an API handler or queue consumer. Recovery re-executes the workflow from the top, using persisted action checkpoints to avoid repeating side effects. This requires deterministic workflow code: clock reads, randomness, and I/O must be placed in actions or explicit checkpoints.

Skipper provides WorkflowTest for isolated in-memory testing with controllable time, so you can test waits, signals, retries, and completion.

Cloud infrastructure diagram showing Skipper's scheduler and database for durable workflows Programming Illustration

Limitations and Considerations

  • At-least-once side effects: Skipper does not make downstream calls transactional. Actions must be idempotent (e.g., deduplicate using stable request IDs).
  • Deterministic workflows: Workflow code must be replay-compatible; avoid non-deterministic operations directly in the workflow body.
  • Operational responsibility: You own the database capacity, recovery, and scheduling for your workflows—no central team manages it.
  • Batch checkpoints: By default, action checkpoints are batched, so a crash may cause several completed actions to re-execute. Immediate checkpointing can narrow this window but doesn't eliminate it.

Skipper is a good fit for Java/Kotlin services that own multi-step processes and want durable execution within their existing deployment. It shines when the alternative is maintaining custom retry jobs, state tables, and polling loops.

For related insights on workflow and observability, see our article on architecting conversational observability for Kubernetes. Also, consider how model retraining strategies can inform decision-making in dynamic systems.

Developer designing workflow classes with state machine DSL in Skipper Development Concept Image

Conclusion and Next Steps

Skipper is now open source with core engine, MySQL/SQLite adapters, test utilities, and an admin UI. The roadmap includes more storage plugins (Postgres, DynamoDB) and improved tooling for workflow evolution and replay debugging.

If you're building Java or Kotlin services with multi-step processes, evaluate Skipper to reduce toil and improve reliability. Start by exploring the GitHub repository and runnable examples. For a deeper dive into the architecture, refer to Airbnb's engineering blog.

Further Reading:

This content was drafted using AI tools based on reliable sources, and has been reviewed by our editorial team before publication. It is not intended to replace professional advice.