Use cases/ Statistics over a 200-sensor array without notebook drift
Per-sensor summary table beside windowed mean-and-uncertainty plots for a 200-sensor array replace with screenshot · 1600 × 900

Statistics over a 200-sensor array without notebook drift

Drop months of CSV logs from all 200 sensors on the canvas, clean and align them onto one clock, then compute windowed means and spreads with real uncertainty. Out comes a set of figures and a per-sensor summary table — and because it is a pipeline, not a notebook, your co-authors re-run it by opening one file instead of guessing which cell to run first.

Intermediate ⏱ 25 min sensorstime-seriesstatisticsuncertaintyreproducibility
#070Drop every sensor's log on the canvas #071Resample every sensor to a common clock #072Roll up windowed stats with uncertainty #073Export the figures and the summary table

Notebook drift is a reproducibility bug#

Two hundred sensors logging for months is exactly the workload that breaks a notebook. Cells run out of order, a cleaning step gets tweaked and never re-applied upstream, and the figure in the draft no longer matches the code that supposedly made it. The data is fine — the process around it is what quietly rots. A pipeline fixes the order by construction, so reopening the canvas always reproduces the same numbers.

Uncertainty is not optional#

A mean with no spread is a claim with no error bar, and reviewers notice. Carrying the standard-error band through every window means the figures ship with their uncertainty attached, not bolted on at the end. Because the window length is a live parameter, you can show a co-author how the trend firms up as the window widens instead of arguing about it.

It grows with the array#

Add sensor 201 next month and its logs drop into the same folder; the canvas picks them up on the next run and the summary table grows a row — no new script, no edits. When a co-author opens the pipeline file, they get the cleaning rules, the alignment clock, and the window settings exactly as you left them, and they re-run the whole analysis from one file.

Step by step

How it's built

  1. #070 · Load 01

    Drop every sensor's log on the canvas

    Drag the archive of long-run CSVs — 200 sensors, months of samples each — onto the canvas. The @load node groups them into one array entity keyed by sensor ID, infers the timestamp and value columns, and previews row counts per sensor. Files parse lazily, so a multi-gigabyte archive opens at once.

    200 sensor logs grouped into one entity with per-sensor row counts replace with screenshot · 1200 × 750
  2. 02

    Reject dropouts and stuck readings

    Add a @shape cleaning node with a rolling MAD outlier check and a stuck-value detector. Bad samples flag in red on the preview and drop from the working series, while the raw stays untouched underneath. Sensor 118's rail-stuck afternoon lights up before you have even scrolled past it.

    Cleaning node flagging outliers and stuck readings across channels replace with screenshot · 1200 × 750
  3. #071 · Shape 03

    Resample every sensor to a common clock

    Chain a sync node and pick a one-minute common grid. Set interpolation for the analog channels and hold-last for the slow ones, and watch 200 ragged timelines snap into one aligned table. That aligned table is a cached intermediate, so every statistic downstream recomputes instantly.

    200 ragged series resampled onto one aligned one-minute grid replace with screenshot · 1200 × 750
  4. #072 · Shape 04

    Roll up windowed stats with uncertainty

    Add a windowed-stats node and compute mean, standard deviation, and a standard-error band per sensor over a rolling window. Expose the window length as a live parameter and drag it; every curve and its uncertainty band rebreathe from the cached aligned table. No rerun, no waiting.

    Windowed mean and standard-error band with a live window-length slider replace with screenshot · 1200 × 750
  5. #073 · Deliver 05

    Export the figures and the summary table

    Finish with a @deliver node. Render a small-multiples figure of the per-sensor trends with their uncertainty bands, and export a per-sensor summary table — count, mean, spread, and window settings — as CSV. The pipeline file ships with it, so a co-author reproduces every number by reopening the canvas.

    Small-multiples figure beside the exported per-sensor summary CSV replace with screenshot · 1200 × 750
The pipeline

Every step used, at a glance

Load

Sensor-array logs

Read long-run CSV logs from all 200 sensors as one lazily-parsed, per-sensor entity.

CSV
Shape

Clean, align, summarise

Reject bad samples, resample onto one clock, and roll up windowed stats with uncertainty.

Deliver

Figures + summary table

Render per-sensor trend figures and export a reproducible summary table.

CSV
200 sensors
summarised in one run
1 clock
every channel aligned to it
Re-runnable
by opening one file, not a notebook

More use cases

All use cases