Dwarf Documentation

Dwarf is a durable execution engine that runs as a library inside your application. It runs agentic and business workflows as graphs of tasks, checkpointing every step to PostgreSQL, with no cluster, broker or control plane to operate. The engine is written in Go, and your tasks can be written in any language: a small host interface decides how each one is reached. It is free and open source under the Apache 2.0 license.

These guides cover how to use it in your own project. For the full API reference, see pkg.go.dev/github.com/microbus-io/dwarf. For the overview, visit www.dwarf.run.

These docs address two readers, and every page names its own at the top:

  • Developers — you are importing Dwarf and building workflows with it.
  • Operators — you are responsible for a deployment that is already running.

A few pages serve both, and say so.

The One-Paragraph Summary

You build a workflow.Graph of tasks and transitions. You implement a Host whose LoadGraph returns a graph by workflow URL and whose ExecuteTask runs one task. The engine creates a flow (one execution of a graph), runs each task in turn, persists state to SQL between steps, follows transitions to decide what runs next, merges parallel branches, and recovers from crashes. You drive it with a handful of operations — Create, Run, Await, Resume, Cancel, Terminate, Continue, Fork — and observe it through logs, metrics, and traces.

For Developers

If you’re new, read in this order:

  1. Getting Started — install, the two-method Host interface, and your first running flow with the in-process test harness.
  2. Concepts — the vocabulary: graph, task, flow, step, reducer, and the flow lifecycle.
  3. How Dwarf Works — what runs where at runtime, how a crash is recovered, and why execution is at-least-once.
  4. Building Graphs — the workflow.Graph API: tasks, transitions, conditions, fan-out, error handling, and reducers.
  5. Writing Tasks — the workflow.Flow carrier: state, control signals, baggage, transient failures and idempotency.

Then dip into the topic guides as you need them:

  • Building a Host — a production host with quotas, caller identity, ownership checks and tenant-scoped listing.
  • Tasks in Other Languages — tasks in Python, TypeScript or any other language, over JSON.
  • Fan-Out & Subgraphs — running work in parallel and calling sub-workflows.
  • Driving Flows — every engine operation: creating, running, inspecting and listing, pausing and resuming, terminating, forking, continuing a flow, retention and expiration, and the error codes each returns.
  • Detecting Completion — the ways to learn a flow’s outcome, and how to make follow-up delivery reliable.
  • Scheduling — priority, fairness, time budgets and workers.
  • Testing — NewEngineUnderTest and hosts.Mux patterns.
  • Patterns — agent loops, human approval, compensation, deferred and recurring work, deadlines and rate limits.

For Operators

Start at Operating Dwarf — what the engine repairs on its own and on what schedule, what genuinely needs a human, and what belongs on a dashboard. Then:

  • Production Checklist — the pre-flight list. Everything that is not defaulted, or where the default is wrong for production, with a link to the guide behind each.
  • Deployment — choosing and tuning a database, declaring shards, connection pools, workers, replicas and shutdown.
  • Observability — structured logs, OpenTelemetry metrics, and distributed tracing.
  • Upgrading — redeploying your app vs. upgrading Dwarf itself (a maintenance window while Dwarf is v0.x), plus the graph-evolution contract that outlives any single deploy.
  • Resharding — adding, retiring and moving shards, and what a shard outage does to cross-shard operations.
  • Backup and Restore — what to back up, and what a restore does to flows.
  • Data Handling & Security — what the engine stores, what search reaches, what telemetry carries, access control, and retention.
  • Inspecting and Repairing a Flow — find a flow, read its history, draw it, and repair it with Fork.
  • Runbook — symptom-indexed incident response: what to do when an alarm fires.
  • Cloud Benchmarks and Sizing & Tuning — the measured capacity of a deployment, and how the engine sizes itself.
  • A Day of Varied Load — one database through 4.5 hours of mixed flows, bursts above its capacity, a rolling restart and a database restart.
  • Dialect Benchmarks — in-repo throughput and latency benchmarks per SQL dialect, and how to run them on your own hardware.

Reference

Next: Getting Started.