Streaming & pipelines#

Align and reshape event streams, and wire operators into a runnable pipeline.

Streams#

Aligning, combining, and reshaping event streams.

Function

Description

CombineLatest

As-of latest-value join of N streams: one row per distinct index (same-index events coalesce).

Delay

Re-stamp each event's index by a fixed time offset (a latency line).

Dropna

Drop events whose value is NaN.

Filter

2-input mask gate - keep each data value whose aligned mask is nonzero.

from_pandas

Convert a pandas Series or DataFrame to a (values, index) tuple.

Merge

Merge N value streams into one index-sorted (values, sources, index).

Resample

Causal windowed downsample of a 1-D or multi-column value stream.

Select

Pick column(s) from a wide (M, N) value stream.

split

Partition a merged tagged stream back into per-source streams.

Stream tuple convention

The (values, index) tuple at every operator boundary.

to_pandas

Convert a (values, index) tuple to a pandas Series or DataFrame.

Pipelines#

Building and running a pipeline.

Function

Description

Pipeline

A reusable N-in / M-out function you define once and call on stored or live data.