Drift Detection¶
See Drift Detectors for behavior and options, and Choosing a Drift Detector for picking one.
Core types¶
Base classes for drift detection in continual learning systems.
This module provides abstract interfaces for drift detectors that can signal when to switch between different learning regimes (continual learning, fine-tuning, retraining).
- class apeiron.drift_detection.detectors.base.LearningRegime[source]¶
Bases:
EnumLearning regime recommendations based on drift severity. - Stable: No significant drift, continue current strategy - Continual Learning: Minor drift, use CL with replay - Fine-Tuning: Moderate drift, fine-tune model - Retrain: Severe drift, retrain from scratch
- STABLE = 'stable'¶
- CONTINUAL_LEARNING = 'continual_learning'¶
- FINE_TUNING = 'fine_tuning'¶
- RETRAIN = 'retrain'¶
- class apeiron.drift_detection.detectors.base.DriftSignal[source]¶
Bases:
objectContainer for drift detection results.
- __init__(regime, drift_detected, drift_score, confidence=None, metadata=None)[source]¶
Initialize drift signal.
- Parameters:
regime (LearningRegime) – Recommended learning regime
drift_detected (bool) – Whether drift was detected
drift_score (float) – Numeric drift severity score (higher = more drift)
confidence (float | None) – Optional confidence in the detection (0-1)
metadata (Dict[str, Any] | None) – Optional additional information
- class apeiron.drift_detection.detectors.base.BaseDriftDetector[source]¶
Bases:
ABCAbstract base class for all drift detectors.
- __init__(name)[source]¶
Initialize drift detector.
- Parameters:
name (str) – Human-readable name for this detector
- abstractmethod update(value, **kwargs)[source]¶
Update detector with new observation and check for drift.
- Parameters:
value (float) – Numeric value to monitor (e.g., loss, accuracy, prediction error)
**kwargs – Additional detector-specific parameters
- Returns:
DriftSignal indicating whether drift occurred and recommended regime
- Return type:
- state_dict()[source]¶
Snapshot the detector’s evolving state so a run can be resumed.
A detector is the one component whose verdict depends on everything it has already seen, so restarting it from scratch mid-stream changes the run’s results rather than merely repeating work. This returns a picklable snapshot of the attributes named in
_STATE_ATTRS.- Returns:
Picklable mapping of state, tagged with the detector class name so
load_state_dict()can reject a mismatched checkpoint.- Raises:
NotImplementedError – If the subclass has not declared
_STATE_ATTRS.- Return type:
- load_state_dict(state)[source]¶
Restore state produced by
state_dict().The detector must already be constructed from the same config that produced the checkpoint; this restores only what the stream accumulated.
- Parameters:
state (Dict[str, Any]) – A mapping previously returned by
state_dict().- Raises:
NotImplementedError – If the subclass has not declared
_STATE_ATTRS.ValueError – If the checkpoint was written by a different detector class.
KeyError – If the checkpoint is missing an expected attribute.
- Return type:
None
Statistical detectors¶
Statistical drift detectors using river library.
These detectors are lightweight, online algorithms suitable for streaming data. They monitor statistical properties of data streams to detect distribution shifts.
References: - river library: https://github.com/online-ml/river
- class apeiron.drift_detection.detectors.statistical_detectors.ADWINDetector[source]¶
Bases:
BaseDriftDetectorAdaptive Windowing (ADWIN) drift detector.
ADWIN maintains a variable-length sliding window and detects change when there’s a significant difference between the means of two sub-windows.
Best for: Detecting gradual and abrupt changes in mean values. Use case: Monitor model loss, accuracy, or prediction errors over time.
- __init__(delta=0.002, minor_threshold=0.3, moderate_threshold=0.6, name='ADWIN')[source]¶
Initialize ADWIN detector.
- class apeiron.drift_detection.detectors.statistical_detectors.KSWINDetector[source]¶
Bases:
BaseDriftDetectorKolmogorov-Smirnov Windowing (KSWIN) drift detector.
KSWIN uses the two-sample Kolmogorov-Smirnov test to compare recent data against a reference window. Detects when distributions differ significantly.
Best for: Detecting changes in data distributions (not just mean). Use case: Monitor prediction distributions, feature statistics.
- __init__(alpha=0.005, window_size=100, stat_size=30, minor_threshold=0.3, moderate_threshold=0.6, name='KSWIN', seed=None)[source]¶
Initialize KSWIN detector.
- Parameters:
alpha (float) – Significance level for KS test (lower = more sensitive)
window_size (int) – Size of reference window
stat_size (int) – Size of recent window for comparison
minor_threshold (float) – Drift score threshold for continual learning
moderate_threshold (float) – Drift score threshold for fine-tuning
name (str) – Detector name
seed (int | None) – Seed for KSWIN’s reference-window sampling. KSWIN draws its reference window at random, so runs are not reproducible unless this is set.
- class apeiron.drift_detection.detectors.statistical_detectors.PageHinkleyDetector[source]¶
Bases:
BaseDriftDetectorPage-Hinkley test for drift detection.
Page-Hinkley detects changes in the mean of a Gaussian signal. It’s fast and works well for detecting abrupt changes.
Best for: Fast detection of abrupt changes in mean values. Use case: Real-time monitoring of model performance metrics.
- __init__(min_instances=30, delta=0.005, threshold=50.0, alpha=0.9999, minor_threshold=0.3, moderate_threshold=0.6, name='PageHinkley')[source]¶
Initialize Page-Hinkley detector.
- Parameters:
min_instances (int) – Minimum instances before detection
delta (float) – Magnitude threshold for change
threshold (float) – Detection threshold
alpha (float) – Forgetting factor (weight of recent values)
minor_threshold (float) – Drift score threshold for continual learning
moderate_threshold (float) – Drift score threshold for fine-tuning
name (str) – Detector name
Model performance detector¶
Model performance-based drift detector using evidently.
This detector monitors model predictions and performance metrics to detect drift. It uses evidently’s data drift detection capabilities for comprehensive analysis.
Reference: https://github.com/evidentlyai/evidently
- class apeiron.drift_detection.detectors.model_performance_detector.ModelEvalDetector[source]¶
Bases:
BaseDriftDetector- __init__(name='ModelEval')[source]¶
Initialize drift detector.
- Parameters:
name (str) – Human-readable name for this detector
- update(value, **kwargs)[source]¶
Update detector with new observation and check for drift.
- Parameters:
value (float) – Numeric value to monitor (e.g., loss, accuracy, prediction error)
**kwargs – Additional detector-specific parameters
- Returns:
DriftSignal indicating whether drift occurred and recommended regime
- Return type:
- class apeiron.drift_detection.detectors.model_performance_detector.ModelPerformanceDetector[source]¶
Bases:
BaseDriftDetectorDrift detector based on model predictions and performance.
Uses evidently to detect drift in: - Prediction distributions - Feature distributions (if features provided) - Target distributions
This is more comprehensive than statistical detectors but requires reference data and batched updates.
- __init__(reference_data=None, reference_predictions=None, reference_targets=None, drift_share_threshold=0.5, minor_threshold=0.3, moderate_threshold=0.6, name='ModelPerformance')[source]¶
Initialize model performance detector.
- Parameters:
reference_data (DataFrame | None) – Reference dataset (features as DataFrame)
reference_predictions (ndarray | None) – Reference model predictions
reference_targets (ndarray | None) – Reference ground truth labels
drift_share_threshold (float) – Share of drifted features to trigger detection
minor_threshold (float) – Drift share for continual learning regime
moderate_threshold (float) – Drift share for fine-tuning regime
name (str) – Detector name
- class apeiron.drift_detection.detectors.model_performance_detector.EnsembleDetector[source]¶
Bases:
BaseDriftDetectorEnsemble of multiple drift detectors with voting mechanism.
Combines signals from multiple detectors to make more robust decisions.
- VOTING_STRATEGIES = {'all': 'unanimous', 'and': 'unanimous', 'any': 'any', 'majority': 'majority', 'or': 'any', 'unanimous': 'unanimous'}¶
- __init__(detectors, voting='majority', name='Ensemble')[source]¶
Initialize ensemble detector.
- Parameters:
detectors (List[BaseDriftDetector]) – List of individual detectors
voting (str) – Voting strategy. One of ‘majority’, ‘any’ (alias ‘or’), or ‘unanimous’ (aliases ‘all’, ‘and’).
name (str) – Detector name
- update(value, **kwargs)[source]¶
Update all detectors and combine signals.
- Parameters:
value (float) – Metric value
**kwargs – Additional parameters passed to individual detectors
- Returns:
Combined DriftSignal
- Return type:
- state_dict()[source]¶
Snapshot every sub-detector, in order.
The ensemble holds no state of its own – its verdict is a vote over the sub-detectors – so the checkpoint is just their states, positionally aligned with
self.detectors.
- load_state_dict(state)[source]¶
Restore every sub-detector from
state_dict().Each sub-detector validates its own class tag, so an ensemble rebuilt with reordered or substituted detectors is rejected rather than silently restored into the wrong slot.
Factory¶
- apeiron.drift_detection.load_drift_detector.load_drift_detector(cfg)[source]¶
Dynamically load and instantiate a drift detector based on its name.
- Parameters:
cfg (Config) – Configuration object containing parameters for the detector.
- Returns:
An instance of the specified drift detector.
- Return type: