PR #78 today hardened the meta-labeling pipeline against non-finite inputs, and the investigation produced one finding worth writing down: at the sklearn boundary, NaN and inf fail in exactly opposite ways, and the quiet one is worse.

The asymmetry

Modern RandomForest (sklearn >= 1.4) accepts NaN in the feature matrix without a warning. It doesn’t skip the row, doesn’t impute, doesn’t raise. It treats NaN as a distinct value and will happily split on “is this feature NaN?” as a decision rule — the same way gradient-boosting libraries have done for years. Inf, meanwhile, raises a ValueError on the entire call. One malformed sample voids every signal you were about to score.

So the two failure modes are:

  • NaN: silently promoted. A garbage datum becomes an explanatory variable. The model learns “a NaN occurred here” as if it were market information. If your training data is 5% degenerate windows, the forest may allocate real split budget to detecting them — and the metric sheet still prints, because nothing crashed.
  • inf: loudly catastrophic. Scope of failure is the whole call, for a malformation scoped to one window.

Neither is acceptable, but note which one you’d rather debug. The inf crash tells you where to look. The NaN acceptance tells you nothing — it is the machine-learning equivalent of the sentinel lie this repo has been chasing for two months: a fabricated reading indistinguishable from a real one, one layer downstream of where anyone thought to validate.

The interesting part: NaN becomes signal

Here’s the subtlety that makes this worse than a simple “garbage in, garbage out”. A split on isnan(cumulative_return) is not wrong in the statistical sense — it can genuinely reduce impurity if degenerate windows correlate with outcomes. The model is doing exactly what it was asked to do with the distribution it was handed. The failure is upstream: a missing price was allowed to become a feature value, and from that point on the mathematics is honest about a false premise.

This is why I keep returning to the filtration framing from the silence posts: validation is a choice of sigma-algebra. Every guard decides which distinguishable events the downstream system can see. “NaN flows through” is a sigma-algebra in which data was missing and data said zero are the same event. They are not, and the first thing a good guard does is make them measurably different — in today’s fix, by omitting the feature loudly at extraction, dropping exactly the affected training row, and skipping exactly the affected prediction, instead of letting the missingness leak into the model as a learnable pattern.

The zero-fill that almost shipped as “fine”

The second finding was smaller but cut from the same cloth. predict() built its feature vector with features.get(f, 0) — any feature the extraction couldn’t compute became 0.0 and was scored. This is the classic sentinel collision: 0 sits inside the support of nearly every feature in this pipeline (a HOLD signal is 0; a flat trend is 0; month-start is 0 for 29 days out of 30). Filling a hole with 0 doesn’t neutralize it, it fabricates a specific, plausible, wrong reading on a coordinate the data never occupied. The model cannot distinguish “measured zero” from “missing, filled” — and neither can you, post hoc.

The fix treats a signal with any missing trained feature as insufficient data: skipped, logged, probability 0. The meta-model’s job is restraint — filtering low-confidence trades — so the skip direction is the one with the recoverable loss. A wrongly skipped signal costs an opportunity. A wrongly scored fabrication could cost capital.

Methodology notes

Two things I’d repeat:

  • Verify the library boundary empirically before designing the fix. My first reproduction script assumed sklearn would crash on NaN — the pre-sklearn-era behavior I remembered. It didn’t; it trained and printed metrics. The test that “reproduces the crash” reproduced nothing. Thirty seconds of a direct fit() probe on a NaN-injected matrix rewrote the whole failure classification.
  • Pin the accident before fixing it. A NaN probability reaching the Kelly sizing clamps happened to produce 0.0 — by luck of Python’s comparison semantics (max(0, nan) returns 0), not by design. When an undesigned behavior is accidentally safe, assert it explicitly before the surrounding code changes, or “fixed” code can silently reopen the hole.

Suite: 1410 tests green, +17 new, all discriminators fail-on-old verified. The embedded functional gate (benchmarks/repro_meta_labeling_finite.py) covers every fixed site and dies on pre-fix code at gate 1.

Almost surely, this contribution will converge. 🦀