Writing workflows
The authoring contract: what a workflow on this engine is, what the engine
guarantees, what a workflow must supply in return, and how to test and
draw one. A human or an agent should be able to write the next workflow
from this page alone. The worked example at the end is example_test.go,
compiled and run by go test, so it cannot drift from the engine.
Vocabulary
| Term | Meaning | In code |
|---|---|---|
| purpose | what a workflow achieves; its workflows are the ways of achieving it, one purpose per workflow | Purpose, on Definition |
| workflow | a kind of work the engine can run, declared once in code | Definition, registered in Definitions |
| revision | a workflow's step list as the code had it for an attempt; the same names in the same order are the same revision | Revision, recorded on RunRecord |
| instance | one run of a workflow, identified within its type by its key | RunRecord; Key for keyed workflows, BusinessDate for date-keyed |
| step | one unit recorded against an instance: a stage, or one account's end of day | StepResult |
| stage | a step of a pipeline as code: a name, a Run, an Until or both, an Undo when its work must not outlive a failed run, and Final when its work must never be redone |
Stage |
| attempt | one call of the runner for an instance; a retry is a new attempt at the same instance, never a new instance | StartedAt is the attempt's start |
| child | an instance started under another, its parent; the parent's children are the instances its sub-process started | RunRecord.ParentID; RunFilter.ParentID lists them, Store.CountDescendants counts them by status at any depth; Store.DeleteRun removes a run and every run under it, for a host deleting the thing the run was about |
| sub-process step | a step that starts instances of a child definition: one child, or children one at a time or all at once | Step.Child, Step.Mode; the stage calls Runner.StartChild |
| mode | how a sub-process step starts its children: one at a time is a loop, the next starting when the previous is through or when an operator starts it; all at once is a fan-out | Mode: OneAtATime, AllAtOnce; unset means one child |
| given | what a child is given, as prose: the engine hands it only its key and business date, so this is the contract of what the key names | Step.Given |
| join | the step whose Until waits for a child-starting step's children to be through |
Step.Joins, naming the child-starting step |
An instance renders from the revision it recorded, not from the live
definition, so it stays right after the code changes: Definition.StepsFor
is its step list, described by the definition where a name still exists.
Declaring a workflow
A definition is the workflow as the registry, the pages and the diagrams
see it. Declare it next to the code that runs it and append it to
Definitions in an init, as gobank-deploy does. The registry starts
empty, so a host's pages show only the workflows it runs: the library
declares DailyUpdate beside RunDaily and leaves registering it to
gobank; the demo app registers that, its own release pipeline, and a
payment batch with its payments as children.
| Field | What it is | Rule |
|---|---|---|
Type |
the workflow's name | lower snake case; unique across every program sharing a store |
Purpose |
what the workflow achieves: a Name and a Description |
share one Purpose value between the workflows that are ways of achieving the same thing |
Description |
what one instance achieves and what a retry does | one or two sentences |
InstanceKey |
what identifies one instance, as prose | "version", "environment@expiry", "business date" |
Steps |
every step the runner will record, in order | Name matches the stage's Name exactly; Description says what it does; Source is an https:// link to the code |
Steps[i].Child, Mode, Given |
the step is a sub-process: it starts instances of the Child definition, one, OneAtATime or AllAtOnce, each Given what the prose says |
the child is a definition of its own, with its own page and picture |
Steps[i].Joins |
the step waits for every child of the named step to be through | name the child-starting step; a fan-out nobody joins is warned about on the page |
Check the registry in the host's tests, as the demo's
definitions_test.go does: every definition has a type, a purpose and a
description, every step links to its code. The instance key is the one
idempotency rule, so choose it from the system under control, not from
the clock: a demo environment is keyed by its expiry, which the server
carries, so a restart finds the same instance.
Stages
A pipeline is Runner.RunPipeline(ctx, type, key, date, stages). Each
stage runs in order and is recorded as a step with the stage's name. A
stage has a Run, an Until, or both: Until is a condition the stage
waits on, and Run executes once it holds. Undo reverses the stage's
work when a later stage fails.
Until |
Run |
Step recorded | Run status | RunPipeline returns |
|---|---|---|---|---|
| unset | returns nil | completed | continues | |
| unset | returns error | failed, with the error | failed | nil, the error |
| false | not called | waiting, once, when the wait began | waiting | the run, nil |
| true | unset | completed | continues | |
| true | returns nil | completed | continues | |
| true | returns error | failed | failed | nil, the error |
| error | not called | failed, with the error | failed | nil, the error |
| completed in an earlier attempt | not called | kept | continues | |
| after the last stage | completed | the run, nil | ||
| key already completed | not called | nil, nil |
When a stage fails, the completed stages before it are undone in reverse
order, nearest first, each by its own Undo; a stage without one keeps
its step and its work.
| Completed stage before the failure | Undo |
Step | The undoing |
|---|---|---|---|
has no Undo |
not called | kept | passes over it |
Undo returns nil |
called | removed | continues to the one before |
Undo returns error |
called | kept, its work may still stand | stops; the error is reported with the failure |
The run's error names the failed stage, then what was undone and any
undo that failed: days: the demo stopped answering (undone: add, create); undo add: delete server: forbidden. Give a stage an Undo when
its work bills or blocks while the run sits failed: a server created, a
lock taken. A tag pushed or a figure recorded is left to stand.
A waiting run is parked, not held: RunPipeline returns, and the host
calls it again on its own tick. The cadence belongs to the host, so there
is no interval on the stage. An approval, a scheme response or a server
coming up is a store write or an observation whose condition is "is it
there yet". Do not sleep and poll inside a stage; put the condition in
Until and let the tick do the polling, so the wait survives a restart.
Resume
A failed instance is retried by calling the runner again with the same
key: the failed step is cleared, the run restarts, and the stages run
again from the first one not completed. A waiting instance is re-entered
the same way, but it continues the same attempt: nothing is cleared and
StartedAt is kept.
| Call for an existing instance | Steps cleared | Run | Stages run |
|---|---|---|---|
after a failure or a crash (failed, running, pending) |
failed, running, pending; the undone ones went when they were undone | RestartRun: new attempt, StartedAt now, the stages' revision recorded |
the first not completed, onwards |
| while waiting | none | ResumeRun once a condition holds, same attempt |
the waiting one, onwards |
| after completion or cancellation | none; the call is a no-op |
This is what resume demands of a stage:
- Idempotent. A stage may run again after a crash that lost its step. Make its effect safe to repeat: create only when absent, push only when not already pushed.
- State on the system under control, not in the run. The run records that a stage completed, nothing more. What the stage made (a server, a tag, a figure) lives where it belongs and is found again from there.
- Resumable from any stage. A later stage cannot assume an earlier one ran in this process. Anything it needs, it reads back.
- Undone when it must not outlive a failure. A stage whose work bills or blocks while the run sits failed has an
Undothat removes it; the next attempt then makes it again. ItsUndois as idempotent as itsRun: deleting what is already gone is not an error. - Its own record, if it has one. A workflow with domain figures to keep (a drill's readings, a performance run's rates) keeps them in its own table against the run's id, found through
FindRunByKey.
Rewind
An operator can send an instance back to an earlier stage with
Runner.Rewind(ctx, type, key, from, stages): when a check finds a problem
that an earlier step must fix. The stage named from and every later one
lose their steps, and the run starts a new attempt; the host's next call to
RunPipeline runs them again, and the stages before from are not
repeated. Rewind does not run anything itself.
| Instance | from |
Rewind | Next RunPipeline |
|---|---|---|---|
any, but no runner on it at the moment (a crash's running is fine) |
a stage, no Final stage completed at or after it |
steps from from on removed; run running, new attempt |
runs from from |
| any | a stage at or before a completed Final stage |
refused | unchanged |
| any | not a stage of the pipeline | refused | unchanged |
| none for the key | — | refused | — |
Mark a stage Final when its work must never be redone, such as an
archive written or money moved. Rewinding after it is allowed. The steps a
rewind removes are not kept as history yet (#5).
Sub-processes
A workflow whose instances have parts with lives of their own — a batch
of loads, a load of photos — is three definitions, not one long one. The
parent's sub-process stage starts each child with Runner.StartChild and
returns; the child is a pending instance of its own definition, which the
host's tick runs by its key like any other. The stage reads back its own
run id with FindRunByKey, as it reads back anything else it needs.
Sub-process stage's Run calls StartChild |
The child key | StartChild |
Then |
|---|---|---|---|
| for a new child | not in the store | creates it: pending, ParentID the parent's, no attempt yet |
the tick's RunPipeline(child type, key, …) finds it by key and runs it, keeping the parent link |
| again, after a crash or a retry | in the store, any status | returns it as it is | the tick carries on with it |
| under a run id the store does not have | — | error | the stage fails |
A stage that starts children all at once never waits for them: a child
whose stage waits on its parent's later stage would deadlock against a
parent waiting on the child. Waiting is a later stage's Until, through
the existing park-and-poll: CountDescendants by status when the rule is
"every child done", or the host's own condition when it is something of
the domain, such as "every photo receipted". That stage names the
child-starting step in Joins, so the definition page and the picture
show where the children are waited for. A stage that starts them one at a
time is a loop, and is its own join unless a later stage is named. The
instance page lists a run's children with their status, and a child's page
links up to its parent. The demo's payment batch (cmd/demo/payments.go)
is the pattern end to end: start payments starts one child per payment,
settle joins them, waiting until none is active and failing the batch if
any failed, and one press of Pay is the host's tick run until the batch
settles.
The engine hands a child only its key and business date (StartChild):
the key names the host's own record, which the child reads. Given on the
child-starting step says what that is, as prose, and the child's definition
page repeats it under Started by.
Mode |
Joins names the step |
Drawn as | The definition page says |
|---|---|---|---|
unset, no Child |
— | a box | — |
OneAtATime |
no | a loop arrow back to the step | starts children one at a time and waits for each: the step is its own join |
OneAtATime |
yes | a loop arrow, a dashed edge into the joining step | starts children one at a time; step waits for all |
AllAtOnce |
yes | stacked boxes, a dashed edge into the joining step | starts a child per item, all at once; step waits for all |
AllAtOnce |
no | stacked boxes | the same, with a warning: nothing waits for the children |
What the store records and what the host supplies
The store records runs and steps: workflow_runs and workflow_steps,
versioned through sqlstore.Migrations. Use sqlstore.New on a pglike
database for real, NewMemStore in tests. A step record is the fact that
the step ran: its name, status, error and duration. It is not what the
step worked on.
Step data lives in the host's tables
The engine never sees a stage's inputs, so it cannot record them, and it
should not: a posting instruction is bank data, and bank data belongs in a
typed table under the bank's schema, not in a payload column on the
engine's. The pattern is the host's own table, in the same database, keyed
to the engine's rows by run_id (what an instance is about) or step_id
(what one step did). A restart with only the database then finds both
where to resume and what to do.
| Data | Lives in | Keyed by | Demo |
|---|---|---|---|
| that a step ran, and how it ended | workflow_steps |
post failed in 1.07ms: posting PAY-0001/3 failed |
|
| what an instance is about | the host's table | run_id |
payments: account, amount, posted_at |
| what one step produced | the host's table | step_id |
an account's day: balances, interest |
| the state the steps change | the host's table, keyed by its own id | accounts: the balance now; the day's steps and the payments move it |
|
| what to do next | the runs (FindRunByKey, the last completed date) |
key | the batch to retry, the date to process |
| anything | process memory | nothing: the simulation's counters are the remaining exception |
Write the row in the stage that first holds the data, and make the write
idempotent on its natural key, since a stage may run again after a crash:
the demo's receive inserts the batch file with ON CONFLICT DO NOTHING,
start payments keys each row to the child run it starts, post stamps
the row posted. Every later stage and the instance page read the row back
rather than carrying a struct from memory: if the struct is the source,
the database is not. The host owns the table's migrations beside the
engine's, in one sequence in gobank, as its own DDL at open in the demo.
The daily step's balance and interest columns on workflow_steps predate
this pattern. They are the exception, not the model.
The host supplies:
| What | Why | |
|---|---|---|
| a store | sqlstore.New(db) or NewMemStore() |
the engine keeps no state of its own |
| its step data | tables keyed by run_id or step_id, as above |
the engine records that a step ran, not what it worked on |
| the tick | a loop that calls the runner until the run is not waiting | the engine never holds a goroutine |
| a log per step | an io.Writer the stages write to |
the step record keeps only name, status, error and duration |
| the pages | instance and definition pages; the demo app and gobank-deploy are two. view.Views.Progress is a run's steps as a row of coloured dots for the host's own page about the thing the run is about; the instance page carries the same row under its title |
the engine renders nothing |
| the clock, injectable | Now and Sleep as fields with real defaults |
tests drive time |
Testing a workflow
Drive one key against NewMemStore() with fakes for the system under
control and an injected clock, then read the store back. runnertest
has the two checks every workflow test makes: OnlyRun is the single
instance, Steps is each step's status by name. A failure test arms one
fake to fail, checks the run is failed with the failed step, then calls
again and checks the same instance completes with the earlier stages not
repeated. A waiting test flips the condition between calls.
A new store implementation runs storetest's contract checks, as
sqlstore and MemStore do.
Drawing a workflow
diagram.Definition writes one d2 file per definition from the registry
(the demo's defd2 does, and task docs:d2 renders them): boxes are the
steps, each linked to its source. A sub-process step is one collapsed box,
BPMN's ⊞ on its name, a loop arrow when it starts children one at a time,
stacked when all at once, linking to the child definition's page; the
child's picture is on that page, and a dashed edge runs from the step to
the one that joins its children. Each definition is drawn twice, across
in one row (diagram.Across, <type>.svg) and down in one column
(diagram.Down, <type>.down.svg); the pages show the one that fits the
screen, down on a phone. Neither is a grid: d2 fills a grid column first
and cannot loop an edge back to a grid cell, and in a row or a column no
edge crosses a step. A long row shrinks to fit the page, which is the
pressure to split: one picture answers one question, so a definition that
does not fit across a screen becomes a sub-process with its own page for
the run of steps that overflows.
diagram.Instance draws one instance with its steps coloured by state;
the demo app serves it drawn across at
workflows/instances/<id>/diagram.d2. Nothing is drawn by hand, so the
pictures cannot drift from the code.
Worked example
A promotion: build, wait for an approval, promote. From example_test.go.
func promotion(approved *bool, log func(string)) []wf.Stage {
return []wf.Stage{
{Name: "build", Run: func(context.Context) error { log("building"); return nil }},
{Name: "approve", Until: func(context.Context) (bool, error) { return *approved, nil }},
{Name: "promote", Run: func(context.Context) error { log("promoting"); return nil }},
}
}
func ExampleRunner_RunPipeline() {
ctx := context.Background()
store := wf.NewMemStore()
approved := false
stages := promotion(&approved, func(s string) { fmt.Println(s) })
runner := wf.NewRunner(store, nil)
date := wf.NewBusinessDate(2026, time.October, 8)
for tick := 1; ; tick++ {
rec, err := runner.RunPipeline(ctx, "promotion", "v0.3.0", date, stages)
if err != nil {
fmt.Println("failed:", err)
return
}
fmt.Printf("tick %d: %s\n", tick, rec.Status)
if rec.Status != wf.StatusWaiting {
break
}
approved = tick == 2 // the approval arrives between the second and third ticks
}
// Output:
// building
// tick 1: waiting
// tick 2: waiting
// promoting
// tick 3: completed
}
The test beside it, TestPromotionRecordsEveryStep, is the testing
pattern above: one key, the store read back through runnertest.
Checklist
- A
Definitionwith type, description, instance key and every step named, described and linked to its code, appended toDefinitions - The instance key comes from the system under control
- Each stage is idempotent and reads back what it needs
- Waits are
Untilconditions, not sleeps - A stage whose work bills or blocks has an
Undo - A stage whose work must never be redone is
Final - A sub-process stage starts its children with
StartChildin itsMode, saying what they areGiven; a later stage'sUntilwaits on them andJoinsit - Each definition fits across one screen; a longer run of steps is a sub-process with its own definition and page
-
NowandSleepare injectable with real defaults - A test for the happy path, one for a failure resumed, one per waiting stage, one for a failure undone where a stage has an
Undo - Domain figures in the workflow's own table against the run id
- The host ticks waiting runs and shows the pages