Concepts

For everyone — developers and operators alike. This is the shared vocabulary the rest of the documentation assumes. Read it once, before the guides.

Five terms run through all of Dwarf: graph, task, flow, step and reducer. This page defines them. What happens at runtime (which process runs what, and how crashes are recovered) is covered next, in How Dwarf Works.

Graph

A graph is the definition of a workflow: which tasks exist, how they connect, and how parallel branches merge. It’s a directed graph of tasks and transitions between them, with an entry point and optional per-field reducers. You build one in code with workflow.NewGraph and the Add* methods; it carries a display name, and your host’s LoadGraph resolves it by a workflow URL you choose. Graphs are reusable templates — many flows run the same graph.

Here is the graph of a trip-planning agent: it plans, searches in parallel, merges what it found, asks a human to approve, and either books or plans again. Dwarf drew it: workflow.NewGraphRenderer(g).Render() turns any graph into a Mermaid chart like this one.

graph LR
    classDef task fill:#32a7c1,color:#f4f2ef,stroke:#32a7c1
    classDef term fill:#e5f4f3,color:#434343,stroke:#32a7c1
    linkStyle default stroke:#32a7c1

    t0["Plan"]:::task
    t1["Search"]:::task
    t2["Draft"]:::task
    t2_reduce(("reduce")):::term
    t2_reduce --> t2
    t3["Approve"]:::task
    t3_switch{"switch"}:::term
    t4["Book"]:::task
    t0 -->|"for each"| t1
    t1 --> t2_reduce
    t2 --> t3
    t3 --> t3_switch
    t3_switch -->|"approved"| t4
    t3_switch -->|"!approved"| t0
    _start(( )):::term --> t0
    t4 --> _end(( )):::term

See Building Graphs.

Task

A task is a named unit of work in a graph. It has a name (its identity within the graph) and a URL (the address your host’s ExecuteTask uses to reach it — a function key, an endpoint, a topic). A task reads inputs from the flow’s state, does its work, and writes outputs back. Tasks are reusable across graphs; the same task URL can appear at several positions.

See Writing Tasks.

Flow

A flow is a single execution of a graph — one running instance. It has a unique key, tracks its current position, and carries a state map that evolves as tasks run. A flow moves through a lifecycle:

graph LR
    classDef state fill:#32a7c1,color:#f4f2ef,stroke:#32a7c1
    classDef final fill:#e5f4f3,color:#434343,stroke:#32a7c1
    linkStyle default stroke:#32a7c1

    _create(( )):::final -->|"Create"| running["running"]:::state
    running -->|"Interrupt"| interrupted["interrupted"]:::state
    interrupted -->|"Resume"| running
    running --> completed["completed"]:::final
    running -->|"task error"| failed["failed"]:::final
    running -->|"Terminate"| terminated["terminated"]:::final
    running -->|"Cancel"| cancelled["cancelled"]:::final

• running — Create makes a flow and runs it right away; its tasks are running or waiting to run.
• interrupted — waiting for outside input, because a task called Interrupt; Resume continues it.
• completed — the last task finished and there was no further transition to follow.
• failed — a task returned an error and the graph had no onError transition for it.
• terminated — stopped immediately by Terminate.
• cancelled — stopped gracefully by Cancel: every loss the flow could not recover from was a cancellation, none a real error.

A flow is open while it is running or interrupted, and closed once it is completed, failed, terminated or cancelled. A closed flow never changes again. About a minute after it closes, it moves to the archive, where it stays readable by key for the retention (SetRetention, 30 days by default) and then is deleted. ListOpen lists open flows and ListClosed archived ones.

A flow’s key is a composite string of the form {shard}-{hour}-{flowID}-{token}: the shard that holds the flow, the hour it was created (counted from 2000-01-01 00:00 UTC), its id on that shard, and a random token. You pass it to every operation.

Step

A step is one task execution within a flow. It records, as JSON:

• state — the input the task saw. It is fixed when the step is created.
• changes — what the task wrote.
• interrupt payload — what the task passed to Interrupt, if it paused the flow.

It also records the step’s status, error, timings and attempt count. The step’s result is its state with its changes applied, each field through its reducer. In a linear chain, that result is the next step’s state:

graph LR
    classDef data fill:#e5f4f3,color:#434343,stroke:#32a7c1
    classDef task fill:#32a7c1,color:#f4f2ef,stroke:#32a7c1
    linkStyle default stroke:#32a7c1

    s1["Step 1 state"]:::data --> t1["Task A"]:::task --> c1["Step 1 changes"]:::data
    s1 --> m(("apply")):::data
    c1 --> m
    m --> s2["Step 2 state"]:::data --> t2["Task B"]:::task

While a task runs, it reads its state with its own writes so far applied. A forEach branch’s state also holds its element. After a fan-out branch’s last step, the result goes to the fan-in, whose state combines every branch’s writes; after the last step of the flow, it is the flow’s final state. Steps are numbered by depth, one deeper than the step they follow, so the parallel branches of a fan-out share a depth.

Because each step’s input is frozen and stored, you can see exactly what any task saw, fork a new flow from any step of one that has ended, and recover from a crash without guessing. The full record of a flow is available with History, and one step with Step.

Continuing a Flow

Continue starts a new flow from a completed one: the new flow inherits the previous flow’s final state and identity, so you can model a chat or an iterative process as a chain of flows that share history. Your application keeps the chain, by continuing the key the last Continue returned.

See Continue.

Reducer

A reducer decides how writes to a state field combine:

A field with no registered reducer uses replace. You wire non-default fields with graph.SetReducer(field, reducer). On a reduced field, every write is a contribution, combined with what the field already holds wherever the task runs. Reading the field returns the combined value:

• A step’s write combines with the value it was handed, and the next step reads the result.
• Parallel branches’ writes combine at their fan-in, in branch order.
• Continue’s additional state combines with the final state of the flow it continues.

So a task writes only its contribution — the message to append, the amount to add — never the value it read, which would count it again. To replace or clear a reduced field, a task calls Overwrite or Del, which are not allowed inside a fan-out branch. See Writing Tasks and Fan-Out & Subgraphs.

How It Fits Together

You define a graph of tasks. The engine creates a flow for each execution, runs one step per task, persisting state between them and combining writes to a field with reducers. Continue starts a new flow from a completed one.

Next: How Dwarf Works.