Data preparation#

Clean a series before analysis: fill or drop missing values, and despike outliers.

Missing data#

Filling and removing NaN values.

Function

Description

Dropna

Drop events whose value is NaN.

Forward-fill NaNs

Replace NaN with the most recent finite value.

Replace NaNs with a constant

Replace NaN with a user-specified scalar.

IsFinite

Returns 1.0 for finite values, 0.0 for NaN or inf. Does not propagate NaN.

IsNan

Returns 1.0 if the input is NaN, else 0.0. Does not propagate NaN.

Outliers & robustness#

Despiking and robust estimators.

Function

Description

Clip to bounds

Bound each element below and/or above.

Hampel filter

Robust Hampel despiker, replace samples far from the window median (in MAD units).

Impulse clip

Causal impulse remover, detects spikes on the trend-free first difference.

Rolling mean absolute deviation

Trailing-window mean absolute deviation from the rolling mean.

Rolling median

Trailing-window median.

Rolling median absolute deviation

Rolling median absolute deviation, median(|x - median|), a robust scale estimate.

Rolling sigma-clip

Replace samples outside [mean - lower*std, mean + upper*std] with NaN or the clipped bound.