Model Harness

The contract every model + data-stream integration implements. See Model Harness for the narrative version.

class apeiron.model.torch_model_harness.BaseModelHarness[source]

Bases: ABC

Members

self.model : nn.Module self.cfg : Dict[str, Any] # e.g., {“device”: “cuda”, …}

You must implement:
  • get_loader(self) -> DataLoader | Iterable

  • get_criterion(self) -> CriterionFn

__init__(cfg, model)[source]
Parameters:
  • cfg (Config)

  • model (torch.nn.Module)

abstractmethod get_optimizer()[source]

Returns the optimizer object compatible with the trainable parameters supports parameter groups for, e.g., different learning rates

Return type:

torch.optim.Optimizer

get_optmizer()[source]

Deprecated misspelled alias for get_optimizer().

Deprecated since version Use: get_optimizer() instead. This alias will be removed in a future release.

Return type:

torch.optim.Optimizer

abstractmethod update_data_stream()[source]

Updates the data stream potentially leading to data drift

Return type:

None

abstractmethod get_stream_dataloader()[source]

Returns a training and validation dataloader compatible with the model input that will be used for continual learning

Return type:

torch.utils.data.DataLoader

abstractmethod get_hist_dataloaders()[source]

Returns a training and validation dataloader with historical data (to measure drift) compatible with the model input If there is no historical data, return None

Return type:

Tuple[torch.utils.data.DataLoader | None, torch.utils.data.DataLoader | None]

get_train_dataloaders()

Returns a training and validation dataloader compatible with the model input that will be used to loop over for inference

Return type:

Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader]

abstractmethod get_criterion()[source]

Return a loss function compatible with model output and dataloader labels

Return type:

Callable[[torch.Tensor, torch.Tensor], torch.Tensor]

eval()

Stream over batches; return mean(metric) over batches (order preserved).

Return type:

List[float]

history_eval()

Stream over batches; return mean(metric) over batches (order preserved).

Returns None if no historical data is available.

Return type:

List[float] | None

register_task(diagonal_metrics)[source]

Record the task just finished so later events can measure forgetting.

A task is one drift event: the window the detector fired on and the CL loop adapted to. This freezes that window’s validation split into a standalone eval set and stores it alongside diagonal_metrics – R[i][i], the score on the window measured right after adapting to it. Both travel in the same record so eviction can never misalign a task’s eval set from its diagonal.

The split is copied into memory rather than referenced: the harness drops its window tensors as the stream advances, and a plain DataLoader reference would keep the whole window alive through a view.

Parameters:

diagonal_metrics (List[float]) – eval() output for the current window, taken after the CL loop finished.

Return type:

None

eval_past_tasks()

Score the current model on every registered task’s frozen eval set.

Returns row T of the train-test matrix below the diagonal – [R[T][i] for i < T], oldest task first, index-aligned with task_diagonals. Empty until at least one task is registered.

Return type:

List[List[float]]

property task_diagonals: List[List[float]]

R[i][i] per registered task, oldest first.

Index-aligned with eval_past_tasks().

property ckpts_enabled: bool
build_checkpoint_payload()[source]

Build the checkpoint object to save.

Subclasses can override this to include additional metadata beyond weights (e.g., preprocessing scalers, feature names, architecture parameters) so that saved checkpoints match the format expected by the loader.

Return type:

By default, returns model.state_dict() (weights only).

save_ckpt(event)[source]

Persist model state, evict oldest when over budget.

Parameters:

event (int)

Return type:

str