Model Harness¶
The contract every model + data-stream integration implements. See Model Harness for the narrative version.
- class apeiron.model.torch_model_harness.BaseModelHarness[source]¶
Bases:
ABCMembers¶
self.model : nn.Module self.cfg : Dict[str, Any] # e.g., {“device”: “cuda”, …}
- You must implement:
get_loader(self) -> DataLoader | Iterable
get_criterion(self) -> CriterionFn
- abstractmethod get_optimizer()[source]¶
Returns the optimizer object compatible with the trainable parameters supports parameter groups for, e.g., different learning rates
- Return type:
torch.optim.Optimizer
- get_optmizer()[source]¶
Deprecated misspelled alias for
get_optimizer().Deprecated since version Use:
get_optimizer()instead. This alias will be removed in a future release.- Return type:
torch.optim.Optimizer
- abstractmethod update_data_stream()[source]¶
Updates the data stream potentially leading to data drift
- Return type:
None
- abstractmethod get_stream_dataloader()[source]¶
Returns a training and validation dataloader compatible with the model input that will be used for continual learning
- Return type:
torch.utils.data.DataLoader
- abstractmethod get_hist_dataloaders()[source]¶
Returns a training and validation dataloader with historical data (to measure drift) compatible with the model input If there is no historical data, return None
- Return type:
Tuple[torch.utils.data.DataLoader | None, torch.utils.data.DataLoader | None]
- get_train_dataloaders()¶
Returns a training and validation dataloader compatible with the model input that will be used to loop over for inference
- Return type:
Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader]
- abstractmethod get_criterion()[source]¶
Return a loss function compatible with model output and dataloader labels
- Return type:
Callable[[torch.Tensor, torch.Tensor], torch.Tensor]
- eval()¶
Stream over batches; return mean(metric) over batches (order preserved).
- history_eval()¶
Stream over batches; return mean(metric) over batches (order preserved).
Returns None if no historical data is available.
- register_task(diagonal_metrics)[source]¶
Record the task just finished so later events can measure forgetting.
A task is one drift event: the window the detector fired on and the CL loop adapted to. This freezes that window’s validation split into a standalone eval set and stores it alongside
diagonal_metrics–R[i][i], the score on the window measured right after adapting to it. Both travel in the same record so eviction can never misalign a task’s eval set from its diagonal.The split is copied into memory rather than referenced: the harness drops its window tensors as the stream advances, and a plain
DataLoaderreference would keep the whole window alive through a view.- Parameters:
diagonal_metrics (List[float]) –
eval()output for the current window, taken after the CL loop finished.- Return type:
None
- eval_past_tasks()¶
Score the current model on every registered task’s frozen eval set.
Returns row
Tof the train-test matrix below the diagonal –[R[T][i] for i < T], oldest task first, index-aligned withtask_diagonals. Empty until at least one task is registered.
- property task_diagonals: List[List[float]]¶
R[i][i]per registered task, oldest first.Index-aligned with
eval_past_tasks().
- build_checkpoint_payload()[source]¶
Build the checkpoint object to save.
Subclasses can override this to include additional metadata beyond weights (e.g., preprocessing scalers, feature names, architecture parameters) so that saved checkpoints match the format expected by the loader.
- Return type:
By default, returns
model.state_dict()(weights only).