How an LSD package fits together

One idea: run and show#

An Engine executes pipelines over a domain Workspace and presents attached views:

• Entities belong to engine.workspace and hold the steps you run.
• Views attach to the Engine and present controls or data.
• Artifacts are disposable execution files under engine.artifact_workspace.

      engine.workspace                               Engine
      entity forest + event bus                      attached Views
      engine.workspace.add(entity)                   engine.add(view)
                  → engine.run(entity.id) / run_all()      → engine.show()

     RUN  (pipelines) ─────────────────────      SHOW  (UI) ─────────────────────────
       Source → LOAD ─Table→ SHAPE ─Table→          View  — ONE class:
                       DELIVER → file, or              • no kind → a form (params + @action)
                       return a ViewerPayload          • a kind  → table · chart ·
       Entity owns the steps + @tool.action                        flowchart · custom
       Template stamps ready-made entities            a view watch(entity)es to get its output

              entity runs ─► output + ViewerPayload ─► ( framework wires the bus ) ─► watching views

Use engine.workspace.add(entity) for entities and engine.add(view) for views. Calling Engine.add with an entity raises an error. Engine() starts a new engine; use Engine.load("project.lsd") to restore a saved project, or Engine(artifact_workspace="./runs") to choose where new execution artifacts go.

The run side — a pipeline#

A pipeline is a chain of typed steps. A Table (a friendly, zero-copy wrapper over Arrow) flows through the ports:

• LOAD — brings data in (source → Table). A Source parameter injects the reader.
• SHAPE — transforms it (Table → Table).
• DELIVER — ends the pipeline. It either writes a result out (Table → file / API) or returns a ViewerPayload (LayerPayload / ChartPayload / LegendPayload) to show it; the framework persists the payload and routes it to any view that watches the entity.

Steps are cached on their parameters: change one knob and only what depends on it recomputes. An Entity owns an ordered set of steps and any right-click actions; a Template turns a dropped file into a ready-wired entity.

from lsdtools import Tool, Table, Engine

tool = Tool("scores")

@tool.load
def numbers(count: int = 5) -> Table:
    return Table({"n": range(count)})

@tool.shape
def doubled(t: Table) -> Table:            # `t: Table` is an input port
    return t.with_column("n", t["n"] * 2)

Engine().run(doubled(t=numbers(count=5))).print()

The show side — a view#

A View is one declarative class, presented as a window when the desktop is present or a console rendering otherwise (so the same code runs headless). It is param() fields plus @action buttons, with optional data:

• with no kind it renders as a form — an interaction surface of params + actions;
• with a kind (table, chart, flowchart, or a custom one) it renders that representation of data.

from lsdtools import views, Engine

engine = Engine()
engine.add(views.chart(kind="histogram", title="Grades"))
engine.show()          # a window, or console text when headless

A View never imports GTK — the desktop registers the renderer and LSD stays front-end-agnostic. The same View a package contributes with @tool.view also works standalone through show().

The coupling — watch(entity)#

Pipeline code does not need a view reference. A view receives an entity's output with one verb, watch. Reusing the numbers and doubled steps above:

from lsdtools import Engine, Entity, views

with Engine() as engine:
    source = numbers(count=5)
    scores = Entity("scores").add(source).add(doubled(t=source))
    engine.workspace.add(scores)
    grid = views.table(title="Scores").watch(scores)
    engine.add(grid)
    results = engine.run_all()
    assert all(result.ok for result in results.values())
    engine.show(mode="text")

view.watch(entity) holds that entity's output and refreshes on every re-run — the framework does the bus wiring (you never call bus.on). Any order works: watch before or after either is added, and watching an entity that has already run pulls its output immediately (which is what makes drag-and-drop feel instant). A deliver step that returns a ViewerPayload is routed the same way — to whatever watches the entity. The desktop's drag-drop, the 3-D checkbox, and right-click "Show in ▸" are all sugar over this one verb.

For live, mid-run signals (progress, a training epoch) steps still ctx.emit custom events — payload return is for the terminal deliver artifact, emit is for transient events while a step runs.

The front door — @tool.*#

You never wire any of this by hand. One Tool object collects the whole package through decorators:

Two rules the model enforces: a step only ever gets a StepContext (the full Context is rejected — it keeps run() pure and cacheable), and the same View has two homes (mounted in the desktop, or shown standalone).

Related#

Start — What is LSD? · Write your first tool

Learn — The Tool object · Naming: which is which

API — Engine · Table · UI overview