# How an LSD package fits together
URL: https://lsd.tools/docs/start-architecture
Updated: 2026-07-18

> An Engine holds two kinds of thing. Entities run — a pipeline of load → shape → deliver steps with a Table flowing through; a deliver either writes a file or returns a ViewerPayload to show a result. Views show — a window or console rendering of a View (a form of params + actions, or a table/chart/flowchart). `engine.add(...)` takes either; `run()` executes the entities, `show()` presents the views. A view `watch(entity)`es to receive that entity's output and refresh on re-runs — the framework wires the bus. Everything is authored with `@tool.*` decorators.

## One idea: run and show

An `Engine` is the whole project. It holds two kinds of thing, and that split is the entire model:

- **Entities** you **run** — a pipeline of steps.
- **Views** you **show** — a piece of UI.

```
                         ┌──────────────────────────────────────────┐
                         │  Engine   (workspace dir · event bus)     │
                         └───────────────┬───────────────┬──────────┘
              engine.add(entity)  ───────┘               └───────  engine.add(view)
                  → run() / run_all()                        → show()

     RUN  (pipelines) ─────────────────────      SHOW  (UI) ─────────────────────────
       Source → LOAD ─Table→ SHAPE ─Table→          View  — ONE class:
                       DELIVER → file, or              • no kind → a form (params + @action)
                       return a ViewerPayload          • a kind  → table · chart ·
       Entity owns the steps + @tool.action                        flowchart · custom
       Template stamps ready-made entities            a view watch(entity)es to get its output

              entity runs ─► output + ViewerPayload ─► ( framework wires the bus ) ─► watching views
```

The symmetry is exact: `engine.add(entity)` then `engine.run()` mirrors `engine.add(view)` then
`engine.show()`. *Everything the Engine holds is either something you run or something you show.*

## The run side — a pipeline

A pipeline is a chain of typed steps. A `Table` (a friendly, zero-copy wrapper over Arrow) flows
through the ports:

- **LOAD** — brings data in (`source → Table`). A `Source` parameter injects the reader.
- **SHAPE** — transforms it (`Table → Table`).
- **DELIVER** — ends the pipeline. It either writes a result out (`Table → file / API`) or
  **returns a [`ViewerPayload`](/docs/api-viewerpayload)** (`LayerPayload` / `ChartPayload` /
  `LegendPayload`) to *show* it; the framework persists the payload and routes it to any view that
  watches the entity.

Steps are cached on their parameters: change one knob and only what depends on it recomputes.
An **Entity** owns an ordered set of steps and any right-click **actions**; a **Template** turns a
dropped file into a ready-wired entity.

```python
from lsdtools import Tool, Table, Engine

tool = Tool("scores")

@tool.load
def numbers(count: int = 5) -> Table:
    return Table({"n": range(count)})

@tool.shape
def doubled(t: Table) -> Table:            # `t: Table` is an input port
    return t.with_column("n", t["n"] * 2)

Engine().run(doubled(t=numbers(count=5))).print()
```

## The show side — a view

A **View** is one declarative class, presented as a window when the desktop is present or a console
rendering otherwise (so the same code runs headless). It is `param()` fields plus `@action` buttons,
with optional data:

- with **no kind** it renders as a *form* — an *interaction surface* of params + actions;
- with a **kind** (`table`, `chart`, `flowchart`, or a custom one) it renders that *representation* of data.

```python
from lsdtools import views, Engine

engine = Engine()
engine.add(views.chart(kind="histogram", title="Grades"))
engine.show()          # a window, or console text when headless
```

A View never imports GTK — the desktop registers the renderer and LSD stays front-end-agnostic. The
same View a package contributes with `@tool.view` also works standalone through `show()`.

## The coupling — `watch(entity)`

The run side and the show side never reference each other. A view gets an entity's output with **one
verb** — `watch`:

```python
from lsdtools import Engine, views

engine = Engine()
engine.add(drillholes(collar="collars.csv", survey="survey.csv"))   # an entity to run
grid = views.table(title="Assays").watch("drillholes")              # a view that follows it
engine.add(grid)
engine.run_all()      # the entity runs → the view holds its output; a re-run refreshes it
engine.show()
```

[`view.watch(entity)`](/docs/api-view-watch) holds that entity's output and refreshes on every re-run —
**the framework does the bus wiring** (you never call `bus.on`). Any order works: watch before or after
either is added, and watching an entity that has already run pulls its output immediately (which is
what makes drag-and-drop feel instant). A [deliver step that returns a
`ViewerPayload`](/docs/tools-rendering) is routed the same way — to whatever watches the entity. The
desktop's drag-drop, the 3-D checkbox, and right-click "Show in ▸" are all sugar over this one verb.

For live, mid-run signals (progress, a training epoch) steps still `ctx.emit` custom events — payload
return is for the terminal deliver artifact, `emit` is for transient events while a step runs.

## The front door — `@tool.*`

You never wire any of this by hand. One `Tool` object collects the whole package through decorators:

| Decorator | Produces |
|---|---|
| `@tool.load` · `@tool.shape` · `@tool.deliver` · `@tool.source` | pipeline steps (get a `StepContext`: progress, cancel, emit) |
| `@tool.entity` · `@tool.template` · `@tool.action` | domain types, stamped entities, entity/menu actions |
| `@tool.view` → `View` | UI surfaces — a sidebar panel, an overlay, or a data view (get the full `Context`: ui, engine, project) |
| `@tool.command` | a `lsd <name>` CLI command |

Two rules the model enforces: a **step** only ever gets a `StepContext` (the full `Context` is
rejected — it keeps `run()` pure and cacheable), and the **same** View has two homes (mounted
in the desktop, or shown standalone).

## Related

**Start** — [What is LSD?](/docs/start-overview) · [Write your first tool](/docs/start-first-tool)

**Learn** — [The Tool object](/docs/tools-tool-object) · [Naming: which is which](/docs/tools-glossary)

**API** — [Engine](/docs/api-engine) · [Table](/docs/api-table) · [UI overview](/docs/ui-overview)

## FAQ
### What is a View?

One declarative class — params plus @action buttons, with optional data (feed/to_text/to_image). A View with no kind renders as a form (params + actions); a View with a kind (table, chart, flowchart, or custom) renders that representation of data. It is declarative and never imports GTK.

### How does a view get a pipeline's output?

It calls `view.watch(entity)` — the one verb linking a view to an entity. The view then holds that entity's output and refreshes on every re-run; the framework does the bus wiring (you never call `bus.on`). A deliver step that returns a ViewerPayload routes to whatever watches the entity. The pipeline never holds a reference to the UI.