Polymorphic input/output behavior#
This page is the exhaustive contract for how a screamer function handles each input type: exactly what you may pass, what you get back, and the dispatch rules that decide. For a gentle, example-led introduction to the same idea, see Using screamer; this page is the reference behind it.
The one property the whole contract exists to guarantee: the same object works on a single scalar, a NumPy array, a strided view, a list, an iterator, or an async generator, so code written against stored data runs unchanged on a live stream.
For combining, splitting, or filtering streams that do not tick together (different rates, async arrival, missing samples), see Streams, values, and alignment. The lockstep contract on this page is the degenerate "no index → row number" case of that model.
The contract has two layers:
1‑input / 1‑output classes (
RollingMean,EwVar,Diff,Lag, everyRolling*, everyEw*, all the math transforms, ...). These inherit fromScreamerBase. The vast majority of screamer.N‑input / 1‑output classes (
RollingCorr,RollingCov,RollingBeta,RollingSpread). These inherit fromFunctorBase<Derived, N, 1>.1‑input / M‑output classes (
RollingMinMax,BollingerBands). These inherit fromFunctorBase<Derived, 1, M>.
The general N‑input / M‑output case is not yet implemented; calling such
a class raises TypeError: Unsupported functor type.
The single-input contract (ScreamerBase)#
Every 1‑in/1‑out class supports the following input/output shapes:
You pass... |
You get back... |
How |
|---|---|---|
|
|
one |
|
|
one |
1D NumPy array, length 1 |
|
unwrapped, treated as scalar |
1D NumPy array, length ≥ 2 |
1D NumPy array, same shape |
one |
1D strided NumPy view |
1D NumPy array, same shape |
strided pass; output is contiguous |
2D NumPy array |
2D NumPy array |
one independent stream per column; |
N‑D NumPy array |
same shape |
|
Python |
NumPy array of same length |
converted to 1D |
Python |
NumPy array of same length |
same |
Python iterator ( |
a screamer |
results yielded one at a time on demand |
Async generator ( |
a screamer |
results awaited one at a time |
Anything else |
|
"Unsupported input type" |
There are two important conventions hidden in this table.
Convention 1. The first axis is time#
For an N‑dimensional array, screamer always treats axis=0 as the time
axis and every other axis as a parallel independent series. Concretely:
import numpy as np
from screamer import RollingMean
# 1D: 100 samples of a single time series
x_1d = np.random.randn(100)
RollingMean(5)(x_1d).shape # (100,)
# 2D: 100 samples of 4 parallel series
x_2d = np.random.randn(100, 4)
RollingMean(5)(x_2d).shape # (100, 4); each column independent
# 3D: 100 samples × 8 instruments × 3 features per instrument
x_3d = np.random.randn(100, 8, 3)
RollingMean(5)(x_3d).shape # (100, 8, 3); 24 independent streams
This means screamer.RollingMean(5)(x) and np.apply_along_axis(...) agree
about the time axis without you ever specifying it. It is a deliberate
simplification: there is no axis= argument because axis=0 is always
the time axis.
If you want the rolling operation to run along a different axis, transpose
the array yourself (x.T) before passing it.
The input shape is preserved exactly. (T,) and (T, 1) are different
inputs and produce different outputs: a 1-D array stays 1-D, a 2-D
column-vector stays 2-D. screamer never silently squeezes or expands
axes.
RollingMean(5)(np.random.randn(100)).shape # (100,)
RollingMean(5)(np.random.randn(100, 1)).shape # (100, 1)
State is cleanly isolated between streams: reset() is called both at
the start of each call and between every column inside it. You can reuse
the same instance for as many calls as you like and they will be
indistinguishable from constructing a fresh instance each time.
Convention 2. Eager for collections, lazy for iterators#
Lists, tuples, and NumPy arrays are processed all at once, in one C++ pass.
Iterators and generators are processed one value at a time: screamer wraps them
in LazyIterator and produces each value only when you advance the iteration.
That separation is what makes the live-event use case work, your
generator can yield from a socket, a Kafka stream, a clock-driven simulator,
and screamer applies the algorithm at exactly the same cadence.
from screamer import RollingMean
mean = RollingMean(5)
# Eager: all 100 values computed up-front, returned as an array
result_eager = mean(np.arange(100.0)) # numpy.ndarray, shape (100,)
# Lazy: same algorithm, results come out one at a time
def stream():
for x in some_live_source():
yield x
for y in mean(stream()): # screamer.LazyIterator
publish(y) # back-pressure preserved
The discriminator is whether the input is a Python list/tuple /
numpy.ndarray (eager paths) or some other iterable (lazy path). A list
is first converted into a NumPy array and then processed at once;
the fact that lists are technically iterable does not matter, the
list/array branch is checked before the iterable branch in the dispatcher.
Async generators are handled symmetrically through LazyAsyncIterator.
Why no lazy NumPy? A NumPy array is already in memory, so eager processing is strictly faster; lazy iteration over an array would buy nothing and pay for
__next__overhead per element.
The dispatch order#
The exact decision tree implemented in ScreamerBase::operator()
(src/screamer/common/base.cpp) is:
Is it a scalar? A scalar is any of:
Python
float,int,bool,NumPy
float32,float64,int32,int64,uint32,uint64.
The check is
can_cast_to_double(obj)ininclude/screamer/common/cast_double.h. If yes, callprocess_scalar(value)and return a Pythonfloat.Is it a NumPy array, list, or tuple? Cast to
numpy.ndarray<float64>. Two sub‑cases:Length 1: extract the single element and treat it as a scalar.
Length ≥ 2: route to
process_python_array, which is the multi‑dimensional handler described in Convention 1.
Is it an iterable? Wrap in
LazyIterator. Iteration is lazy; eachnext()advances the source and produces one output.Is it an async generator? (
hasattr(obj, "__aiter__")andhasattr(obj, "__anext__").) Wrap inLazyAsyncIterator.Anything else →
TypeError("Unsupported input type for call: ...").
Some concrete consequences of this order#
obj([1, 2, 3])returns a NumPy array of length 3, not aLazyIterator, because lists are matched in step 2 before reaching the iterable branch.obj([1])returns afloat, not a length‑1 array.obj(iter([1, 2, 3]))returns aLazyIterator, becauseiter(...)is iterable but not a list, tuple, or array.obj(np.float32(2.5))returns a Pythonfloat, because NumPy scalars are recognised in step 1.obj(decimal.Decimal("1.5"))raisesTypeError.Decimalis not in the scalar table; we deliberately avoid silent precision conversion.8‑bit and 16‑bit NumPy integers are also not in the scalar table. If this becomes annoying for a use case, raise an issue, adding them is a one‑line change in
cast_double.h.
The multi-input contract (FunctorBase<_, N, 1>)#
Multi-input classes accept the same conceptual shapes, scalars, arrays,
streams, but in N parallel slots. RollingCorr(window_size) is the
reference example with N = 2.
You pass... |
You get back... |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
same shape; |
|
works; result is contiguous |
|
|
|
|
Mismatched shapes or ndim across the |
|
Mixed kinds (one scalar + one array, etc.) |
|
This is the array form of obj(A[:, 0], A[:, 1], ...), and is what makes
RollingCorr(w)(CombineLatest()(a, b)[0]) work directly.
from screamer import RollingCorr
corr = RollingCorr(window_size=20)
# Scalar pair (returns a float)
corr(1.5, 2.0)
# Two parallel arrays (returns an array)
returns_a = np.diff(np.log(price_a))
returns_b = np.diff(np.log(price_b))
corr(returns_a, returns_b)
# Streaming list of pairs (returns a list)
ticks = [(1.5, 2.0), (1.6, 2.1), (1.4, 1.9), ...]
corr(ticks)
# Two parallel generators
for c in corr(gen_a(), gen_b()):
publish(c)
Multi-D semantics for paired arrays#
The 2D / N-D rules are the obvious generalisation of the single-input
convention: axis 0 is time across both inputs, every higher axis is
treated as independent paired streams. Concretely, for two (T, K)
arrays:
corr = RollingCorr(window_size=10)
X = np.random.randn(100, 4) # 4 series of x
Y = np.random.randn(100, 4) # 4 series of y, paired with X column-wise
result = corr(X, Y) # shape (100, 4)
# result[:, k] == corr(X[:, k], Y[:, k]) for every k (bit-exact)
The pairing is column-by-column: X[:, k] is correlated against
Y[:, k] only. Cross-pair correlations (X[:, j] vs Y[:, k] for
j != k) are not computed, that would be a different operator
returning a (T, K, K) result and is out of scope for RollingCorr.
The streams are independent: reset() is called between columns inside
a single batch call, just like for the single-input array path.
Caveats specific to multi-input#
The
Nparallel iterables case is eager, not lazy: the helper builds astd::vector<double>of all results before returning. There is noLazyIteratorat the moment forN > 1. If you want truly streaming multi-input processing today, feed the values yourself in a loop:corr = RollingCorr(window_size=20) for x, y in zip(stream_a, stream_b): yield corr(x, y)
(Each scalar call is constant time and matches the dispatcher's first row in the table.)
All
Ninputs must be the same kind (all scalars, all numpy arrays, all iterables) and the same shape. Mixing them raisesTypeError.
The multi-output contract (FunctorBase<_, 1, M>)#
Some functions produce more than one value per time step. RollingMinMax
returns the pair (min, max); BollingerBands returns the triple
(lower, mid, upper). The call shape mirrors the single-input one with
one rule added: the output gets an extra trailing axis of size M.
You pass... |
You get back... |
|---|---|
scalar |
Python |
1-D array of shape |
NumPy array of shape |
2-D array of shape |
NumPy array of shape |
N-D array of shape |
NumPy array of shape |
iterable |
|
The shape rule is exactly: output.shape == input.shape + (M,). The same
input-axis-preservation guarantee from the single-output path holds here
too. (T,) and (T, 1) produce different outputs: (T, M) and (T, 1, M)
respectively.
from screamer import RollingMinMax, BollingerBands
RollingMinMax(5)(np.random.randn(100)).shape # (100, 2)
RollingMinMax(5)(np.random.randn(100, 4)).shape # (100, 4, 2)
BollingerBands(20)(np.random.randn(100)).shape # (100, 3)
BollingerBands(20)(np.random.randn(100, 4)).shape # (100, 4, 3)
Per-step access uses the trailing axis: bb[:, 0] is the lower band,
bb[:, 1] the mid, bb[:, 2] the upper. For a 2-D input, it would be
bb[:, k, 0], bb[:, k, 1], bb[:, k, 2] for each parallel series k.
Like the multi-input path, the iterable case is eager: it returns
list[tuple[...]], not a lazy iterator.
The multi-input multi-output contract (FunctorBase<_, N, M>)#
Functions that map N parallel input streams to M parallel output streams compose the two rules above: inputs are paired column-by-column (from the N → 1 path) and outputs gain a trailing axis of size M (from the 1 → M path). Cart2Polar and Polar2Cart are reference examples with N = M = 2.
You pass... |
You get back... |
|---|---|
|
tuple of |
|
tuple of |
|
|
|
NumPy array of shape |
|
NumPy array of shape |
|
|
|
NumPy array of shape |
The shape rule is exactly: output.shape == single_input.shape + (M,). Mismatched shapes across the N inputs raise TypeError, same as N → 1. The dispatcher calls reset() between independent paired streams in a 2D/N-D batch, so stateful N → M functors don't leak state across columns.
from screamer import Cart2Polar, Polar2Cart
# Two scalars -> tuple of two floats
Cart2Polar()(3.0, 4.0) # (5.0, 0.9272...)
# Two 1D arrays -> shape (T, 2)
Cart2Polar()(np.random.randn(100), np.random.randn(100)).shape # (100, 2)
# Two 2D arrays -> shape (T, K, 2), paired column-by-column
Cart2Polar()(np.random.randn(100, 4), np.random.randn(100, 4)).shape # (100, 4, 2)
# Roundtrip: Polar2Cart and Cart2Polar are inverses
polar = Cart2Polar()(x, y)
back = Polar2Cart()(polar[:, 0], polar[:, 1]) # equals (x, y)
Symmetry table#
The same algorithm produces the same numbers across every input shape
(modulo NaN/precision noise around warmup):
Property |
Holds? |
|---|---|
|
✓ exact (modulo IEEE float reorderings during reductions, which are not done) |
|
✓ exact |
|
✓ exact |
Same instance, called twice on the same data |
✓ identical output (regression-tested in |
Reusing an instance vs. constructing a fresh one |
✓ identical (one |
State, reset, and causality#
All state lives on the instance.
reset()zeroes it out.The eager array paths call
reset()at the start of every call and at the end. So passing a NumPy array is functionally equivalent to constructing a fresh instance for that batch.Between independent series in a 2D/3D array,
reset()is called automatically, columnkdoes not see columnk‑1's state.The lazy iterator path does not call
reset()for you. The whole point of streaming is preserving state across__next__calls. If you want to start over, construct a new instance or callinstance.reset()yourself.Causal. Output at index
tdepends only on inputs at indices<= t; no function looks ahead.
NaN and warmup documents warmup, the leading region where a
function has not yet seen enough samples, and the start_policy argument
("strict", "expanding", "zero") that controls it.
Design notes#
The single polymorphic dispatch is the load-bearing element of the API, for four reasons:
One mental model for two runtimes. A backtest reads from a pandas DataFrame; a live system reads from a queue, a websocket, or a clock-driven simulator. Both route to the same C++ inner loop, because the dispatcher is the only place that touches the input surface.
Numbers are bit-exact across shapes. There is no training/serving skew: if live values disagree with a backtest, the cause is a difference in the data feed, not the library.
Adding an algorithm costs nothing on the dispatch side. A new class implements
process_scalar(double)(and optionally the array fast paths), and inherits all input shapes.The multi-dimensional convention is explicit. There is no
axis=argument to forget: time is always axis 0, everything else is parallel.
See also#
tests/test_io_size.py, exhaustively walks every shape for every algorithm and asserts the output shape and dtype are correct.tests/test_view.py, strided‑view correctness across the full library.tests/test_stream_vs_batch.py, proves the eager (array) path and the scalar path produce the same numbers.tests/test_stream_vs_generator.py, proves the eager array path and the lazy generator path produce the same numbers.tests/test_rolling_corr.py, the multi-input contract forN=2.