lsdtools reference

lsdtools — the **public, stable SDK** for building LSD packages.

Install it into your project's local Python and import everything from here::

    pip install lsdtools
    ...
    from lsdtools import Tool, Engine, Table

The name is deliberately distinct from ``lsd``: if a module imports ``lsdtools``
it is using the **public** API; if it imports anything under ``lsd.*``
(``lsd.core``, ``lsd.flow``, ``lsd.runtime`` …) it is reaching into LSD
**internals**, which are private and may change without notice. So a quick scan
of a package's imports tells you, at a glance, whether it stays on the supported
surface.

This is a small, curated façade: a handful of authoring names (``Tool``,
``Context``, ``StepContext``, the ``View`` interaction model), the data facade
(``Table``, ``Source``), lazy model-query helpers (``models``), deterministic
geostatistical estimation (``geostats``), authenticated neural execution
(``neural``), headless integer-agent decisions (``integeragent``), the
interop/standalone objects (``Entity``, ``Engine``, ``RunResult``), typed
failures, plus the AI bridge (``generate_step``). The *extension seams* — the component registries the charter puts
on every view kind (``register_chart_kind`` / ``register_cell_renderer`` /
``register_param_control`` / ``register_layer_kind``), the symbology seams
(``register_style_asset`` / ``register_default_style``), the data-format registry
(``register_table_reader``), and the raw step/param/event substrate the ``@tool.*`` API
sits on — live in :mod:`lsdtools.extend`, a **declared, charter-enforced** surface you
reach for when the declarative API can't express something.

Authoring vs. running
---------------------
You author against ``lsdtools`` in any local Python (for editor/AI context and,
if you like, local runs). Runtime selection is deliberately not an SDK capability: the native Hub
resolves a project's exact version pin into its authenticated per-user runtime store. Package tests
install an exact package snapshot into an isolated ``LSD_HOME`` and launch the project through the
supported product/testkit entry point; they never search Program Files or execute a guessed runtime.

Quick start
-----------
    from lsdtools import Tool, Engine, Table

    tool = Tool("hello")

    @tool.load
    def numbers(count: int = 3) -> Table:
        return Table({"x": range(count)})

    @tool.shape
    def double(t: Table) -> Table:
        return t.with_column("x", t["x"] * 2)

    if __name__ == "__main__":
        Engine().run(double(t=numbers(count=3))).print()   # [{'x': 0}, {'x': 2}, {'x': 4}]

API_VERSION = '1.0'

One toolkit-neutral field in a complete View-action input document.

``json`` is the escape hatch for exact arrays and objects whose deeper domain validation
belongs to the action handler. Hidden fields hold immutable versions or review fences and
therefore must also be read-only.

name: str

label: str

kind: str

description: str

description = ''

visibility: str

visibility = 'visible'

read_only: bool

read_only = False

choices: tuple[Any, ...]

choices = ()

minimum: float | int | None

minimum = None

maximum: float | int | None

maximum = None

step: float | int | None

step = None

validate(self, value: Any) -> Any

self: (unannotated)
value: Any

Return one immutable, detached value after exact kind/range validation.

to_wire(self) -> dict[str, Any]

self: (unannotated)

Return the exact toolkit-neutral 1.0 field declaration.

A closed 1.0 form whose submission is one complete immutable JSON object.

fields: tuple[ActionInputField, ...]

submit_label: str

submit_label = 'Apply'

version: str

version = '1.0'

validate(self, value: Any) -> FrozenDict

self: (unannotated)
value: Any

Validate an exact complete submission and detach it into immutable JSON.

to_wire(self) -> dict[str, Any]

self: (unannotated)

Return the exact toolkit-neutral 1.0 schema.

ChartPayload(self, *, kind: str='', title: str='', tables: Optional[Dict[str, Any]]=None, spec: Optional[Dict[str, Any]]=None, **extra: Any) -> None

A chart delivered to a chart viewer (``viewer/chart/set``).

Spec-only by default: ``ChartPayload(kind="bar", title="Grades", series=[...])`` folds everything
into ``spec`` and carries no tables (pass ``tables=`` if a chart is table-backed).

topic = 'viewer/chart/set'

__init__(self, *, kind: str='', title: str='', tables: Optional[Dict[str, Any]]=None, spec: Optional[Dict[str, Any]]=None, **extra: Any) -> None

extra: Any
kind: str
self: (unannotated)
spec: Optional[Dict[str, Any]]
tables: Optional[Dict[str, Any]]
title: str

Chunk(self, view: 'TripView', index: int, offset: int, stop: int) -> None

One row range of a view: ``c["x"]`` reads, ``c.out["x"]`` writes, ``c.reduce`` folds.

__init__(self, view: 'TripView', index: int, offset: int, stop: int) -> None

index: int
offset: int
self: (unannotated)
stop: int
view: 'TripView'

count(self) -> int

self: (unannotated)

eid(self) -> 'np.ndarray'

self: (unannotated)

__getitem__(self, name: str) -> Any

name: str
self: (unannotated)

reduce(self, key: str, value: Any) -> None

key: str
self: (unannotated)
value: Any

Record a per-chunk partial; the scheduler folds partials in chunk order.

index: int

index = index

offset: int

offset = offset

stop: int

stop = stop

out: (unannotated)

out = _ChunkOut(view, view._resolved_writes(), offset, stop)

One typed column of a Trip.

``dtype`` is the storage type and is fixed for the component's life; ``kind`` is the profile
kind it snapshots under (defaults from the dtype); ``shape`` is ``()`` for a scalar or
``(k,)`` for a fixed-width vector (``fixed_size_list<dtype>[k]`` in Arrow, an ``(n, k)`` numpy
view); ``mirror_of`` names the components this one is a derived, differently-typed image of
(a float32 render mirror of float64 coordinates) so its staleness is checkable;
``field_id`` is the project field identity minted once at the first materialization and
carried through every snapshot.

name: str

dtype: str

kind: str

kind = ''

shape: Tuple[int, ...]

shape = ()

frozen: bool

frozen = False

mirror_of: Tuple[str, ...]

mirror_of = ()

unit: Optional[str]

unit = None

field_id: Optional[str]

field_id = None

numeric(self) -> bool

self: (unannotated)

width(self) -> int

self: (unannotated)

Values per row: 1 for a scalar, ``k`` for a ``(k,)`` vector.

with_field_id(self, field_id: str) -> 'Component'

field_id: str
self: (unannotated)

element_field_id(self, index: int) -> str

index: int
self: (unannotated)

The field identity of element *index* of a vector component.

A vector snapshots as ``k`` scalar columns (the profile layer measures scalars); their
identities derive from the component's own id, not from whichever step happened to
snapshot it, so they are stable for the component's whole life.

element_name(self, index: int) -> str

index: int
self: (unannotated)

The snapshot column name of element *index* (``name.0``, ``name.1``, ...).

to_dict(self) -> Dict[str, Any]

self: (unannotated)

from_dict(cls, data: Mapping[str, Any]) -> 'Component'

cls: (unannotated)
data: Mapping[str, Any]

Configurable(self, *, id: Optional[str]=None, metadata: Optional[Dict[str, Any]]=None, event_bus: Optional['EventBus']=None, **kwargs: Any) -> None

Base class for all configurable objects: Steps, Entities, Engine.

Subclasses declare parameters with ``param()``; serialization, validation,
introspection, and event emission are inherited automatically.

type_id: ClassVar[str]

type_id = ''

serialized_fields: ClassVar[frozenset[str]]

serialized_fields = frozenset()

param_group_order: ClassVar[List[str]]

param_group_order = []

serialized_type_id(self) -> str

self: (unannotated)

Stable type identity written to and restored from project data.

A live variant instance keeps the identity of the configurable it
replaces.  Consumers that bind declarations to project nodes must use
this public coordinate rather than the concrete provider class or the
private deserialization marker.

serialized_field_names(cls) -> frozenset[str]

cls: (unannotated)

Exact persisted fields understood by this class in the current schema.

register_variant(cls, identity: str, provider: str, klass: type, mode: str='replace') -> None

cls: (unannotated)
identity: str
klass: type
mode: str
provider: str

Register *klass* (from *provider*) as a variant of node *identity* (see the note above).

resolve_class(cls, type_name: Optional[str], *, variant_state: 'Optional[Dict[str, dict]]'=None) -> Optional[type]

cls: (unannotated)
type_name: Optional[str]
variant_state: 'Optional[Dict[str, dict]]'

Resolve an exact type under an explicit or scoped project selection.

variant_providers(cls, identity: str) -> 'List[str]'

cls: (unannotated)
identity: str

Provider keys that offer a variant of *identity* (empty if none).

validate_variant_state(cls, state: 'Optional[Dict[str, dict]]') -> 'Dict[str, dict]'

cls: (unannotated)
state: 'Optional[Dict[str, dict]]'

Validate and normalize one project's exact variant selection.

variant_scope(cls, state: 'Optional[Dict[str, dict]]') -> Generator[None, None, None]

cls: (unannotated)
state: 'Optional[Dict[str, dict]]'

Apply one validated project selection only within the current task/thread.

unregister(cls, name: str) -> bool

cls: (unannotated)
name: str

Remove an exact type id from the registry. Returns True if it was present.

Used on package reload/disable so step/entity classes deleted from a
package's source disappear from ``available_steps()`` instead of lingering.
Any variant implemented by the removed class is purged as well.

__init__(self, *, id: Optional[str]=None, metadata: Optional[Dict[str, Any]]=None, event_bus: Optional['EventBus']=None, **kwargs: Any) -> None

event_bus: Optional['EventBus']
id: Optional[str]
kwargs: Any
metadata: Optional[Dict[str, Any]]
self: (unannotated)

set_parameter(self, name: str, value: Any, *, coerce: bool=False) -> List[ValidationError]

coerce: bool
name: str
self: (unannotated)
value: Any

Set a parameter by name. Returns error list (empty = success). Never raises.

set_parameters(self, values: Dict[str, Any], *, coerce: bool=False) -> List[ValidationError]

coerce: bool
self: (unannotated)
values: Dict[str, Any]

Validate and apply several parameters atomically.

Every name, coercion, and value is checked before the first mutation.
On success values are committed in mapping order and ordinary callers
receive one batch-change event; on validation failure the instance is
unchanged and no event is emitted.  When invoked inside an existing
:meth:`batch_update`, changes join that outer batch.

get_parameter(self, name: str) -> Any

name: str
self: (unannotated)

set_output(self, name: str, value: Any) -> None

name: str
self: (unannotated)
value: Any

Store a computed output value from within run() without firing change events.

Safe to call from background threads. Use for output=True params only.

get_outputs(self) -> Dict[str, Any]

self: (unannotated)

Return {name: value} for all output=True params.

Called by the runner after step execution to include in STEP_FINISHED event data.
Any subscriber (UI, CLI, logging) can read computed results from that event.

get_parameters(self) -> List[ParameterInfo]

self: (unannotated)

Snapshot of all parameters for inspector / CLI generation.

parameter_groups(self) -> OrderedDict

self: (unannotated)

Parameters grouped by their ``group`` value.

Returns an ``OrderedDict[group_name, list[ParameterInfo]]``.
Groups appear in ``cls.param_group_order`` order first, then in
first-seen order for any groups not listed there.

parameter_schema(cls) -> Dict[str, Any]

cls: (unannotated)

Return a JSON Schema ``object`` for all non-hidden, non-output parameters.

Used by agents to build and validate tool call payloads without source inspection::

    schema = MyStep.parameter_schema()
    # → {"type": "object", "properties": {...}, "required": [...]}

validate(self) -> ValidationResult

self: (unannotated)

Validate all parameters. Returns structured result, never raises.

batch_update(self) -> Generator[None, None, None]

self: (unannotated)

Suppress per-field events; emit one batch event on exit.

to_dict(self) -> Dict[str, Any]

self: (unannotated)

validate_serialized_record(cls, data: Dict[str, Any], *, path: str='configurable', validate_parameters: bool=True) -> None

cls: (unannotated)
data: Dict[str, Any]
path: str
validate_parameters: bool

Validate the current persisted contract without constructing an object.

from_dict(cls, data: Dict[str, Any]) -> 'Configurable'

cls: (unannotated)
data: Dict[str, Any]

Restore one validated current-schema record.

set_metadata(self, key: str, value: Any) -> None

key: str
self: (unannotated)
value: Any

remove_metadata(self, key: str) -> None

key: str
self: (unannotated)

clear_metadata(self) -> None

self: (unannotated)

get_metadata(self, key: str, default: Any=None) -> Any

default: Any
key: str
self: (unannotated)

Context(self, *, host: Any=None, engine: Any=None, project: Any=None, entity: Any=None, selection: Any=None, ui: Any=None, viewer: Any=None, view_id: Optional[str]=None, panel: Any=None, events: Any=None) -> None

The single injected object interactive handlers receive (actions / commands / panels).

Fields are populated lazily by the runtime for the surface that invoked the handler; unset ones
are ``None``. ``ctx.viewer`` is an exact-target facade over standard serializable viewer actions;
it never exposes the host's Desktop or renderer implementation. Declare the Context by
annotating any parameter ``Context`` (the name is up to you)::

    @tool.action("entity/csv_import/settings")
    def change_settings(ctx: Context):
        ctx.ui.notify("re-importing with the new settings")
        ...

Use :meth:`testing` to build a stub for unit tests.

__init__(self, *, host: Any=None, engine: Any=None, project: Any=None, entity: Any=None, selection: Any=None, ui: Any=None, viewer: Any=None, view_id: Optional[str]=None, panel: Any=None, events: Any=None) -> None

engine: Any
entity: Any
events: Any
host: Any
panel: Any
project: Any
selection: Any
self: (unannotated)
ui: Any
view_id: Optional[str]
viewer: Any

testing(cls, **fields: Any) -> 'Context'

cls: (unannotated)
fields: Any

Build a Context stub for tests: ``Context.testing(project=fake_project, ui=fake_ui)``.

host: Any

host = host

engine: Any

engine = engine

project: Any

project = project

entity: Any

entity = entity

selection: Any

selection = selection

ui: Any

ui = ui

viewer: Any

viewer = viewer

view_id: Optional[str]

view_id = view_id

panel: Any

panel = panel

events: Any

events = events

Creates(self, *components: Component) -> None

Components a system adds to the Trip (each a :class:`Component` declaration).

components: Tuple[Component, ...]

__init__(self, *components: Component) -> None

components: Component
self: (unannotated)

FIXED = 'fixed'

REFERENCE_FIELD = 'reference-field'

PROJECT_ENVELOPE = 'project-envelope'

ROBUST_PERCENTILE = 'robust-percentile'

PER_TABLE = 'per-table'

Explicit authority for the data range used by one style assignment.

mode: DomainMode | str

minimum: float | None

minimum = None

maximum: float | None

maximum = None

reference_field_id: str | None

reference_field_id = None

contributor_field_ids: tuple[str, ...]

contributor_field_ids = ()

lower_percentile: float | None

lower_percentile = None

upper_percentile: float | None

upper_percentile = None

version: int

version = 1

fixed(cls, minimum: float, maximum: float) -> 'DomainPolicy'

cls: (unannotated)
maximum: float
minimum: float

per_table(cls) -> 'DomainPolicy'

cls: (unannotated)

from_dict(cls, data: Mapping[str, Any]) -> 'DomainPolicy'

cls: (unannotated)
data: Mapping[str, Any]

to_dict(self) -> dict[str, Any]

self: (unannotated)

Engine(self, workspace: Optional[Workspace]=None, *, artifact_workspace: 'Optional[str | os.PathLike[str]]'=None, name: Optional[str]=None, **kwargs: Any) -> None

Execution runtime over a Workspace-owned entity forest.

``param_group_order`` keeps project-level settings in a logical tab order
for the desktop inspector.

param_group_order = ['Project', 'Execution']

project_path: Optional[str]

desktop_runtime: Optional[Dict[str, Any]]

viewer_provider_bindings: Dict[str, Dict[str, str]]

viewer_provider_state: Dict[str, Dict[str, dict]]

viewer_layout: Dict[str, list]

name = param(str, default='project', label='Name', group='Project', validators=[_name_not_empty])

artifact_workspace = param(str, required=True, label='Artifact workspace', group='Project', widget='dir', description='Execution artifact root; each entity gets a sub-folder')

max_parallel = param(int, default=1, min=1, label='Max Parallel', group='Execution', visibility='advanced', description='Max simultaneous pipeline runs; >1 runs independent entity branches concurrently in run_all')

__init__(self, workspace: Optional[Workspace]=None, *, artifact_workspace: 'Optional[str | os.PathLike[str]]'=None, name: Optional[str]=None, **kwargs: Any) -> None

artifact_workspace: 'Optional[str | os.PathLike[str]]'
kwargs: Any
name: Optional[str]
self: (unannotated)
workspace: Optional[Workspace]

workspace(self) -> Workspace

self: (unannotated)

The domain entity forest and EventBus owner consumed by this engine.

rejected_package_contract(self) -> Optional[str]

self: (unannotated)

Why this Engine is an inspectable, non-writing recovery session.

set_parameter(self, name: str, value: Any, *, coerce: bool=False) -> List[Any]

coerce: bool
name: str
self: (unannotated)
value: Any

Keep artifact path writes wrapped in :class:`ArtifactWorkspace`.

execution_status(self) -> ExecutionStatusSnapshot

self: (unannotated)

Return a frozen, runtime-only snapshot of current/recent execution.

The snapshot contains no live Entity/Step objects and mutating a dict
returned by its :meth:`to_dict` helper cannot affect this Engine.

entity_execution_gate(self, entity: Entity)

entity: Entity
self: (unannotated)

Hold this Engine's per-entity execution gate for one live Entity.

The public form of :meth:`_execution_lock_for` for direct callers of
the low-level ``runner.run_pipeline`` — its concurrency note requires
that callers provide the same serialization boundary Engine uses,
because a pipeline mutates its manifest, output paths, and last-run
fields (see ``runner.py`` "Concurrency note"). Same Entity: at most one
execution at a time; distinct entities never block each other.

The gate is a plain (non-reentrant) lock, and pipeline events are
emitted synchronously while it is held — a bus handler must never call
``run``/``run_branch``/this gate for the SAME entity, or it deadlocks.

add(self, view: Any) -> 'Engine'

self: (unannotated)
view: Any

Attach a :class:`~lsd.core.view.View` presented by :meth:`show`.

Entity ownership and mutation belong to :attr:`workspace`.

views(self) -> List[Any]

self: (unannotated)

All attached views, in insertion order.

remove_view(self, view: Any) -> 'Engine'

self: (unannotated)
view: Any

show(self, *, mode: str='auto', title: Optional[str]=None, block: bool=True) -> Any

block: bool
mode: str
self: (unannotated)
title: Optional[str]

Present every attached view — a single window hosting them all when a front-end is loaded,
else each view's titled text rendering. The symmetry partner of :meth:`run_all`.

table_profiles(self) -> 'Dict[tuple[str, str, str], Any]'

self: (unannotated)

Latest derived tables indexed by exact producer coordinate.

Keys are ``(entity_id, step_id, output_port)``. No column-name or
display-name merge is performed; observed fields absent from the
project semantics catalog remain visible as unbound observations.

table_profile(self, entity_id: str, step_id: str, output_port: str) -> Any

entity_id: str
output_port: str
self: (unannotated)
step_id: str

Return one exact current profile, or ``None`` when unmaterialized.

run(self, target: 'str | Entity | Any', *, resume: bool=True, from_step: Optional[str]=None, cancellation_signal: Any=None, run_capabilities: Any=None) -> 'PipelineResult | Any'

cancellation_signal: Any
from_step: Optional[str]
resume: bool
run_capabilities: Any
self: (unannotated)
target: 'str | Entity | Any'

Run a pipeline.

*target* may be:

* a stable **entity id**, a unique entity name, or an **Entity** instance —
  returns a :class:`~lsd.runtime.runner.PipelineResult`;
* a **wired step** or **list of steps** (``engine.run(double(t=numbers()))``) —
  the steps and their upstream closure are wrapped in an implicit entity and run,
  returning a friendly :class:`~lsd.runtime.runner.RunResult` (``.output`` is a
  :class:`~lsd.table.Table`, plus ``.print()``).

``cancellation_signal`` is an optional callable or Event-like signal.
It is checked at runner boundaries and exposed to package code through
``StepContext.cancelled``.

run_branch(self, target: 'str | Entity', terminals: Any, *, resume: bool=True, from_step: Optional[str]=None, cancellation_signal: Any=None, run_capabilities: Any=None) -> 'PipelineResult'

cancellation_signal: Any
from_step: Optional[str]
resume: bool
run_capabilities: Any
self: (unannotated)
target: 'str | Entity'
terminals: Any

Run attached terminal steps and their same-entity ``bind_from`` closure.

The existing entity is passed directly to the canonical runner, so step
ownership, workspace, manifest, and event bus are preserved. References
to steps owned by other entities are treated as already-materialized
external inputs and are never executed or re-parented.

Only explicit ``bind_from`` edges define this branch. Implicit sequential
predecessors are not inferred; a branch that needs one must bind it.
Execution still follows the runner's normal Load -> Shape -> Deliver order.

set_variant(self, identity: str, *, mode: str='replace', provider: Optional[str]=None) -> int

identity: str
mode: str
provider: Optional[str]
self: (unannotated)

Choose the active provider for a node *identity* in this project and swap live nodes.

``mode="replace"`` makes *provider* win the identity (existing nodes instantly use it);
``mode="coexist"`` reverts the identity to the original (each override stays addable on its
own). Persisted in the ``.lsd`` (``package_state.variants``). Returns the number of live
steps swapped.

resolve_configurable_class(self, identity: Optional[str]) -> Optional[type]

identity: Optional[str]
self: (unannotated)

Resolve a type using this project's variant selection only.

variant_scope(self)

self: (unannotated)

Scope nested classmethod deserialization to this Engine's exact selection.

disabled_packages(self) -> 'frozenset[str]'

self: (unannotated)

Packages whose steps are frozen for this project (explicit per-project disables).

Steps owned by these packages serve their cached output and refuse to recompute, so a run
neither loses them nor executes stale/withheld code.

set_runtime_parallelism(self, value: Optional[int]) -> None

self: (unannotated)
value: Optional[int]

Override ``max_parallel`` for this session only (``None`` clears it).

A runtime dial, not a project setting: the ``max_parallel`` param is
untouched, so ``to_dict()``/``save()`` output stays byte-identical and
the override never persists into the ``.lsd`` document.

effective_max_parallel(self) -> int

self: (unannotated)

The parallelism ``run_all`` actually uses: the session override when
set, else the persisted ``max_parallel`` param.

run_all(self, *, resume: bool=True, cancellation_signal: Any=None) -> 'Dict[str, PipelineResult]'

cancellation_signal: Any
resume: bool
self: (unannotated)

Run every entity in dependency order (sources before dependents).

``max_parallel == 1`` (default): strictly sequential.
``max_parallel > 1``: independent branches execute concurrently on a
thread pool; a failed entity's descendants are *blocked* (not launched)
while unrelated branches continue.

``resume=True`` skips unchanged entities via the manifest (incremental
rebuild — composes correctly with both modes).

``cancellation_signal`` stops admitting later entities and is leased to
every active step for cooperative cancellation.

Raises ``CycleError`` if the dependency graph contains a cycle.

run_upstream_of(self, target: 'str | Entity', *, resume: bool=True, cancellation_signal: Any=None) -> 'Dict[str, PipelineResult]'

cancellation_signal: Any
resume: bool
self: (unannotated)
target: 'str | Entity'

Run *target* and the transitive closure of entities it depends on.

Useful for targeted rebuilds: if ``report`` depends on ``raw``, then
``engine.run_upstream_of(report.id)`` runs ``raw`` then ``report``
while leaving unrelated entities untouched. Results are keyed by stable
entity id.

``cancellation_signal`` stops admitting later entities and is combined
with the enclosing project task's lifetime signal when one is present.

plan(self, target: 'str | Entity | None'=None) -> Dict[str, Any]

self: (unannotated)
target: 'str | Entity | None'

Dry-run preview: what would execute without running anything.

Returns a serialisable dict describing which steps would run vs. be
skipped (cached). Reads the manifest and current parameters; no writes.

    engine.plan()               # all entities
    engine.plan(sales.id)       # one entity by stable id
    engine.plan(my_entity)      # one entity by instance

diff(self, target: 'str | Entity | None'=None) -> 'List[Dict[str, Any]]'

self: (unannotated)
target: 'str | Entity | None'

Return steps whose parameters changed since the last successful run.

Compares current param values against the ``param_snapshot`` stored in
the manifest after each successful execution.  Returns an empty list when
nothing has changed, or the manifest has no prior snapshot.

    changes = engine.diff("sales")
    # → [{"entity_name", "step_id", "step_class",
    #     "current_params", "old_params"}, ...]

close(self) -> None

self: (unannotated)

Release resources owned by this engine. Safe to call repeatedly.

__enter__(self) -> 'Engine'

self: (unannotated)

__exit__(self, *_exc: object) -> None

_exc: object
self: (unannotated)

save(self, path: Optional[str]=None, *, document_sections: Optional[Mapping[str, Any]]=None) -> None

document_sections: Optional[Mapping[str, Any]]
path: Optional[str]
self: (unannotated)

Serialize the whole project (engine params + all entities) to JSON.

The first explicit *path* becomes this document's save target; later calls may omit it.
Records ``packages`` as explicit activation roots. Package-owned entity records carry
transitive/implicit provenance themselves. ``document_sections`` lets the application add
opaque,
JSON-compatible top-level sections (for example Desktop's ``"ui"`` block) to this same
atomic snapshot. Such sections cannot replace fields owned by the engine.

load(cls, path: str, *, project_document: Optional[Mapping[str, Any]]=None, workspace: Optional[Workspace]=None, artifact_workspace: 'Optional[str | os.PathLike[str]]'=None, package_manager=None, package_host=None, packages_preloaded: bool=False, inactive_packages: Optional[Dict[str, str]]=None, rejected_package_contract: Optional[str]=None) -> 'Engine'

artifact_workspace: 'Optional[str | os.PathLike[str]]'
cls: (unannotated)
inactive_packages: Optional[Dict[str, str]]
package_host: (unannotated)
package_manager: (unannotated)
packages_preloaded: bool
path: str
project_document: Optional[Mapping[str, Any]]
rejected_package_contract: Optional[str]
workspace: Optional[Workspace]

Restore an engine from a JSON file saved by ``engine.save()``.

Loads the project's declared ``packages`` first so their step/entity classes
are registered before deserialization (a no-op if already loaded).

``workspace`` supplies the domain forest/EventBus owner. ``artifact_workspace``
overrides the serialized execution-artifact directory when a caller needs a
location specific to the current checkout or process.

``project_document`` supplies an already-decoded strict snapshot. The caller retains
ownership of that mapping, so loading works from a deep copy and never rereads ``path``;
the path remains the runtime save/document identity. Omitting it preserves ordinary file
loading exactly.

``packages_preloaded`` is the explicit Desktop handoff: package activation was already
attempted with the Desktop PackageHost, so only metadata is restored here and discovery/
loading is not repeated. Headless callers retain the default load behavior.

``rejected_package_contract`` is a fail-closed Desktop recovery handoff. It is accepted only
with preloaded mode and an ``inactive_packages`` map covering every recoverable package name;
exact/variant state is then ignored and package-owned nodes restore as placeholders.

ephemeral(cls, *, workspace: Optional[Workspace]=None) -> 'Engine'

cls: (unannotated)
workspace: Optional[Workspace]

Engine backed by temporary artifact storage for an optional Workspace.

Entity(self, name: str, *, id: Optional[str]=None, traits: Optional[Set[str]]=None, metadata: Optional[Dict[str, Any]]=None, event_bus: Optional['EventBus']=None, **kwargs: Any) -> None

A domain node in a Workspace entity hierarchy.

``Entity("name").add(LoadStep(...)).add(ShapeStep(...)).run()``

``**kwargs`` in ``__init__`` accepts declared entity parameters such as
``workspace_dir`` and parameters added by subclasses. Entity classes define
persisted domain state and behaviour only. A package
makes one available to authoring surfaces by mounting an
:class:`lsd.packages.EntityTypeSpec` (normally through ``@tool.entity``).
Labels, icons, creation availability, and tree presentation never live on
the entity model.

serialized_fields = frozenset({'traits', 'configurables', 'child_entities'})

name = param(str, required=True, label='Name', description='Entity name', validators=[_name_not_empty])

workspace_dir = param(str, default=None, visibility='advanced', label='Workspace', widget='dir', description='Directory for intermediate Parquet files')

__init__(self, name: str, *, id: Optional[str]=None, traits: Optional[Set[str]]=None, metadata: Optional[Dict[str, Any]]=None, event_bus: Optional['EventBus']=None, **kwargs: Any) -> None

event_bus: Optional['EventBus']
id: Optional[str]
kwargs: Any
metadata: Optional[Dict[str, Any]]
name: str
self: (unannotated)
traits: Optional[Set[str]]

workspace(self) -> Any

self: (unannotated)

The domain workspace that owns this entity, including descendants.

engine(self) -> Any

self: (unannotated)

The currently associated execution Engine, if the workspace has one.

add(self, configurable: Configurable) -> 'Entity'

configurable: Configurable
self: (unannotated)

configurables(self) -> List[Configurable]

self: (unannotated)

All attached configurables, in display order.

step(self, reference: str) -> Optional[Configurable]

reference: str
self: (unannotated)

Find one child by stable id or an unambiguous human alias.

    entity.step("clean_csv")     # a @tool.load def clean_csv  → class CleanCsv
    entity.step("CleanCsv")      # by class name

An exact id always wins. Class and builder-function aliases are
conveniences only: if more than one child matches, use a stable step id.

parent(self) -> 'Optional[Entity]'

self: (unannotated)

The entity that contains this one, or ``None`` if top-level.

child_entities(self) -> 'List[Entity]'

self: (unannotated)

Direct child entities (separate from step configurables).

add_entity(self, child: 'Entity') -> 'Entity'

child: 'Entity'
self: (unannotated)

Attach *child* as a sub-entity.

Raises ``ValueError`` on a cycle (adding an ancestor or self).
An attached parent asks its Workspace to adopt the complete subtree,
removing an existing root from the workspace roots atomically.
Returns ``self`` for chaining.

remove_entity(self, child: 'Entity') -> None

child: 'Entity'
self: (unannotated)

Detach *child* and its complete subtree from this workspace.

move_to(self, new_parent: 'Entity') -> None

new_parent: 'Entity'
self: (unannotated)

Re-parent this entity under *new_parent*.

Raises ``ValueError`` if *new_parent* is this entity or a descendant
(which would create a cycle).

is_ancestor(self, other: 'Entity') -> bool

other: 'Entity'
self: (unannotated)

Return ``True`` if *other* is an ancestor of this entity (i.e. in the parent chain).

iter_descendants(self) -> 'Generator[Entity, None, None]'

self: (unannotated)

Depth-first pre-order traversal of all descendant entities.

output(self) -> 'Any'

self: (unannotated)

This entity's terminal output as an :class:`lsd.table.Table`, or ``None`` if it has not run.

The last configurable that recorded an output path wins (steps run in order, so that is the
entity's terminal result); its ``"output"`` key is preferred, else its first. Reads the Parquet
the run wrote — so it survives a project reopen, where
``Engine._restore_manifest_outputs`` repopulates ``last_output_paths`` from the manifest.

This is the pull side of :meth:`lsd.core.view.View.watch`: a watching view reads it on
``engine/entity/finished`` and whenever it starts watching.

output_at(self, reference: 'Any', output_port: 'Optional[str]'=None, *, expected: str='table') -> 'Any'

expected: str
output_port: 'Optional[str]'
reference: 'Any'
self: (unannotated)

Read one exact declared step output, or ``None`` when it is unrun/stale.

``reference`` is preferably an :class:`lsd.OutputRef`; ``step_id`` plus
``output_port`` is accepted as concise sugar.  Resolution is deliberately
local and exact: the referenced entity must be ``self``, the configurable
id must identify one of this entity's pipeline steps, and the port must be
declared by that step.  No terminal, first-path, or similarly named port
fallback is performed.

The default ``expected="table"`` reads only a Parquet table.  Use
``expected="path"`` when an explicitly declared port is intentionally a
non-table artifact; a ViewerPayload JSON sidecar never masquerades as a
table.

A Trip-valued port (``lsd.trip``; its recorded value is a ``mem://``
handle) answers ``expected="trip"`` with the resident :class:`lsd.trip.Trip`
(restored from its snapshot when it is no longer resident), and answers
``expected="table"``/``"path"`` with its parquet *snapshot* -- the table
form a run leaves behind -- or ``None`` until the runner has written one.

declared_output_refs(self, output_port: 'Optional[str]'=None) -> 'List[Any]'

output_port: 'Optional[str]'
self: (unannotated)

Enumerate exact declared pipeline outputs in configurable order.

``output_port`` is an optional exact filter.  This is the public,
domain-neutral discovery companion to :meth:`output_at`: callers must
still reject zero or multiple matches when their contract requires one.

viewer_payloads(self) -> 'List[Dict[str, Any]]'

self: (unannotated)

The persisted viewer payloads (``viewer/layer/set`` / ``chart`` / ``legend`` …) this entity's
deliver steps produced — reconstructed generically, with no per-step code.

For each configurable, this reads the ``*_payload.json`` sidecar a :class:`ViewerPayload`
deliver wrote (found among its :attr:`last_output_paths`), rebases the stored Parquet paths onto
the sidecar's own directory (robust to a moved workspace), and stamps the *current* entity/step
ids. So a watched entity's layers reappear after a project reopen — where the runtime cache is
gone but the workspace survives (``Engine._restore_manifest_outputs`` repopulates
``last_output_paths``) — without re-running.

The sole mechanism is the JSON sidecar: every deliver step that returns a :class:`ViewerPayload`
writes a ``*_payload.json`` beside its Parquet outputs. Empty until the entity runs.

has_viewer_steps(self) -> bool

self: (unannotated)

True if any step *can* deliver a viewer payload — i.e. its class carries a
``viewer_payload_type`` (stamped by ``@tool.deliver`` on a ``-> ViewerPayload`` function).

A **class**-level check (no import of any domain package, no run needed): it answers before the
entity has run, where :meth:`viewer_payloads` is still empty. That is what lets a front-end
accept a drop of an un-run entity onto a viewer — the viewer
:meth:`~lsd.core.view.View.watch`\es it and fills in once the run finishes.

table_profiles(self) -> 'Dict[str, Dict[str, Any]]'

self: (unannotated)

Latest derived profiles by exact ``step_id`` then ``output_port``.

Profiles remain separate observations. They are never merged by an
editable column name, so equally named fields from multiple outputs
cannot silently acquire one another's identity or statistics.

input_entity_deps(self) -> Set[str]

self: (unannotated)