Dropna

Drop events whose value is NaN. This changes the length of the stream (it is a cardinality-changing stream operator, unlike the shape-preserving compute functors). Usable eagerly and inside a Pipeline.

Feeding a lazy iterator of (value, index) pairs returns a lazy iterator of the surviving events; feeding arrays or (values, index) tuples returns the batch result.

Example

The event at index 2 is NaN, so it is removed; the surviving values are returned values-first.

import numpy as np
from screamer import Dropna

vals = np.array([1.0, np.nan, 3.0])
idx  = np.array([1, 2, 3])

clean_vals, clean_idx = Dropna()(vals, index=idx)
print(clean_vals)
print(clean_idx)

[1. 3.]
[1 3]