Load steps

A load step is where data enters a pipeline. It produces a Table and takes no Table input ports — it is the source, so there is nothing upstream to read.

Usage#

The smallest load step builds a table from its parameters:

from lsdtools import Tool, Table

tool = Tool("demo")

@tool.load
def rows(n: int = 3) -> Table:
    """Emit a tiny table of n rows."""
    return Table({"i": list(range(n)), "sq": [i * i for i in range(n)]})

n: int becomes a number field on the node. Run it → a 2-column table. # → i:[0,1,2] sq:[0,1,4]

Reading files with Source#

To let the user pick real data, add a parameter annotated Source. LSD injects a reader bound to whatever the user selects (a file, a folder + glob, or a SQL query); call source.read() to get a Table:

from lsdtools import Tool, Source, Table

tool = Tool("io")

@tool.load(source_kinds=("file", "sql"))
def load_data(src: Source) -> Table:
    """Load a CSV/Parquet file or a SQL query the user picks."""
    return src.read()

source_kinds narrows the picker (default is file). A Source parameter is only valid on @tool.load — using it on a shape/deliver step raises at import time.

Emitting several tables#

Pass outputs=[...] to produce named output ports; return a dict. Downstream nodes wire to a specific port with step.output("name"):

@tool.load(outputs=["nodes", "edges"])
def load_graph(src: Source) -> dict:
    tbl = src.read()
    return {"nodes": nodes_of(tbl), "edges": edges_of(tbl)}

Parameters vs ports — the rule#

Inside any step signature:

• a Table annotation → an input port (load steps may not have these);
• a Source annotation → an injected reader (load only);
• a Context / StepContext annotation → injected capabilities, excluded from the cache key;
• anything else → a typed parameter (a form field; no default ⇒ required).

These are checked at import time, so mistakes surface when you load the package, never mid-click.

Related#

Learn — Shape steps · Deliver steps · Params vs ports

API — Tool · Source · Table

Examples — File loader · Multi-output mesh