Deliver steps

A deliver step is where a pipeline exits. It takes a Table input port and either performs a side effect — writes a file, runs an export — or returns a ViewerPayload to show a result. It is the counterpart to a load step: one ends the graph, the other begins it. The symmetry is deliberate — a load takes a src: Source (the framework injects the reader), and a deliver returns a payload (the framework persists and routes it).

Two shapes of deliver#

from lsdtools import Tool, Table, ChartPayload

tool = Tool("io")

@tool.deliver
def to_csv(t: Table, path: str = "out.csv") -> None:
    """A side effect: write the table out. Returns nothing."""
    t.write_csv(path)

@tool.deliver
def grades_chart(t: Table) -> ChartPayload:
    """Show a result: return a payload the framework persists and routes to the viewer."""
    ys = [r["grade"] for r in t.to_pylist()]
    return ChartPayload(kind="bar", title="Grades",
                        series=[{"name": "grade", "x": list(range(len(ys))), "y": ys}])

t: Table is the input port; path is a parameter. The return annotation decides the shape: a deliver that returns None is a pure side effect; one annotated -> ViewerPayload (or a subclass) is a viewer-delivering step. A ViewerPayload return is only valid on @tool.deliver — a load or shape step returns a Table.

Writing files#

Use the Table writers — t.write_csv(path) and t.write_parquet(path, compression="snappy"). Rather than hard-code a location, take a StepContext and write under ctx.output_dir, the per-step directory the engine manages:

from lsdtools import Tool, Table, StepContext
import os

@tool.deliver
def export_parquet(t: Table, name: str = "result", ctx: StepContext = None) -> None:
    """Write a Parquet file into the step's managed output directory."""
    path = os.path.join(ctx.output_dir, f"{name}.parquet")
    t.write_parquet(path)
    ctx.emit("file/written", path=path, rows=len(t))

ctx.scratch_dir is the place for throwaway intermediates. ctx is injected and excluded from the cache key.

Showing a result — return a ViewerPayload#

To show something, a deliver returns a ViewerPayload. You do not hand-roll an event or write Parquet yourself. On a returned payload the framework:

1. writes each table in the payload's tables to Parquet in the step's workspace;
2. persists the spec + table paths + routing + stamped ids as a *_payload.json sidecar;
3. stamps entity_id / step_id centrally; and
4. emits the payload's topic.

On reopen the sidecar is reconstructed generically — no per-step code. Any View that watches the entity receives the output and refreshes; the 3-D viewer and chart HUDs subscribe to the topics too. Rendering & payloads covers the routing rule in full.

The three built-ins map to the three viewer topics:

from lsdtools import Tool, Table, LayerPayload

tool = Tool("geo")

@tool.deliver
def points(t: Table) -> LayerPayload:
    """Push a point layer to the 3-D viewer — no ctx, no emit, no parquet-write by hand."""
    return LayerPayload(kind="points", tables={"path": t}, style={"color": "grade"})

LayerPayload(kind=..., tables=..., style=..., **extra) folds kind / style (and any extra keyword) into the payload's spec; ChartPayload(kind=..., title=..., series=..., **extra) and LegendPayload(title=..., colormap=..., **extra) are spec-only by default. For a viewer topic of your own, subclass ViewerPayload, set its topic, and register_payload it — see ViewerPayload.

Routing knobs#

A viewer-delivering step gains three optional "Viewer" parameters — layer, view, and target — auto-injected onto the node. Leave them blank for the default (an auto entity:step layer id, the current in-scene view, the active viewer); set them to direct the payload at a specific layer, an in-scene view, or a named main-view target.

ctx.emit is for live, mid-run events — not for showing a result#

ctx.emit(topic, **data) still exists, and is the right tool for live, mid-run signals — progress ticks, a training epoch, a "file written" notice — that a view or the run report listens for. It is not how a deliver shows its terminal artifact: returning a ViewerPayload is, because the framework persists and reconstructs it across a reopen, which a bare emit does not. Rule of thumb: a payload return is for the finished deliver artifact; ctx.emit is for transient events while a step runs.

Deliver steps are terminal#

A deliver node sits at the end of a chain — it has an input port but nothing downstream wires to a returned table (a returned ViewerPayload is a side artifact, not a wired output). If you need to both transform and pass data on, use a shape step; reserve deliver for exits, side effects, and shown results.

Related#

Learn — Rendering & payloads · Load steps · Shape steps

API — Tool.deliver · ViewerPayload · View.watch · Table

Examples — Clean a CSV · Live node metrics