Continuous Learning¶
This document describes the continual-learning path triggered after drift detection.
Main Components¶
ContinuousTrainerinsrc/apeiron/training/continuous_trainer.pyUpdater factory
create_updater(...)insrc/apeiron/training/updater/create_updater.pyUpdater implementations in
src/apeiron/training/updater/
Training Loop Flow¶
When ContinuousMonitor detects drift:
ContinuousTrainer.outer_cl_training_loop(...)is called.Current and historical loaders are pulled from the model harness.
Selected updater runs
cl_preprocessing().For each step up to
cfg.train.max_iter, trainer runsinner_cl_training_loop(...).Each inner step does:
optimizer.zero_grad()updater.update_pre_fwd_bwd()repeated
updater.fwd_bwd(...)for gradient accumulationupdater.update_post_fwd_bwd()optimizer.step()updater.update_post_optimizer_call()
Updater runs
cl_postprocessing()once at end.
train Config Keys Used By CL¶
Defined in TrainCfg (src/apeiron/config/configuration.py):
Key |
Default |
Meaning |
|---|---|---|
|
required |
Per-loader batch size; also used as minimum accepted batch in |
|
required |
DataLoader worker count used by harness loaders. |
|
required |
Optimizer learning rate used by provided harnesses. |
|
|
Number of forward/backward micro-steps before one optimizer step. |
|
|
Max CL optimizer updates per drift event. |
continual_learning Config Keys¶
Defined in ContinualLearningCfg:
Key |
Default |
Meaning |
|---|---|---|
|
|
Updater selection key. |
|
|
Mix historical replay data into |
|
|
SAM parameter-perturbation radius ( |
|
|
Perturbation radius on the historical inputs along the drift direction; |
|
|
|
|
|
EWC regularization strength ( |
|
|
EMA decay for online Fisher prior in EWC. |
|
|
KFAC penalty strength ( |
|
|
EMA decay for running Kronecker factors in KFAC mode. |
Updater Modes (update_mode)¶
base -> BaseUpdater¶
File:
src/apeiron/training/updater/base.pyBehavior: plain supervised forward/backward on current batch only.
Extra config: none.
jvp_reg -> JVPRegUpdater¶
File:
src/apeiron/training/updater/jvp_reg.pyThe JVP updater now implements a first-order Sharpness-Aware / Bertsimas robust update rather than the original Jacobian-Vector-Product regularization term.
Each step:
Computes the current-batch gradient and uses its normalized negation
u_newas the SAM parameter-perturbation direction.Builds one combined batch, half current and half historical, and shifts the historical inputs by
jvp_rho_xalong the unit batch-mean drift directionunit(mean(X_cur) - mean(X_hist)), signed byjvp_data_sign.Back-propagates the combined loss evaluated at
theta + jvp_rho_theta * u_new, then restores the parameters. Only first-order information is used – no Hessian and no third-order terms.
Keeping the current batch inside the loss every step anchors online accuracy, so adaptation does not trade it away for retention.
Relevant config:
continual_learning.jvp_rho_thetacontinual_learning.jvp_rho_x(set to0for parameter-only SAM)continual_learning.jvp_data_sign
Manages the historical batch itself, so it ignores
mix_historic_data.If no historical batch is available, it falls back to base update behavior.
ewc_online -> OnlineEWCUpdater¶
File:
src/apeiron/training/updater/ewc.pyKeeps running parameter anchor (
theta_star) and diagonal Fisher estimate.Adds EWC gradient penalty before optimizer step.
Updates Fisher/anchor once per CL event in
cl_postprocessing().Relevant config:
continual_learning.ewc_lambdacontinual_learning.ewc_ema_decay
kfac_online -> OnlineKFACUpdater¶
File:
src/apeiron/training/updater/kfac.pyTracks layer-wise activation/gradient statistics via hooks.
Applies KFAC-structured EWC-like penalty.
Supports modules:
nn.Linearnn.Conv2d
Relevant config:
continual_learning.kfac_lambdacontinual_learning.kfac_ema_decay
none -> NoUpdater¶
File:
src/apeiron/training/updater/no_updater.pyfwd_bwd(...)is a no-op and returns-1.0.Useful for disabling CL gradient updates while keeping monitoring flow intact.
CL Trigger Conditions¶
CL is only dispatched when detector output has drift_detected = True. Drift checks depend on drift_detection settings:
detection_interval > 0enables periodic checks.detection_interval <= 0disables checks and therefore disables CL dispatch in current monitor logic.
Minimal CL Config Example¶
[train]
batch_size = 64
num_workers = 4
init_lr = 0.001
grad_accumulation_steps = 2
max_iter = 600
[continual_learning]
update_mode = "jvp_reg"
jvp_rho_theta = 0.05
jvp_rho_x = 1.0
jvp_data_sign = 1.0
ewc_lambda = 1000.0
ewc_ema_decay = 0.95
kfac_lambda = 1e-2
kfac_ema_decay = 0.95
Codes wanting to do continual learning should use the ContinuousTrainer class that takes the configuration parameters, model harness, logger and the profiler.
cfg = build_config ( argv )
xtrainer = ContinuousTrainer(
cfg=cfg,
modelHarness=modelHarness,
logger=logger,
profiler=flops_profiler,
)
trainer.outer_cl_training_loop(drift_event_id=drift_count)
if modelHarness.ckpts_enabled:
ckptpath = modelHarness.save_ckpt(event=drift_count)