Apeiron Configuration Reference¶
This document describes all currently supported TOML configuration options available in Apeiron.
The code to parse the configuration file can be found in src/apeiron/config/configuration.py.
The parser expects these TOML sections:
Required:
[model],[data],[train],[drift_detection]Optional:
[continual_learning],[visualization],[logging]Required top-level keys:
seed,deviceCommon top-level optional key:
verbosity
See each section for field-by-field meaning and defaults.
Root-Level Configuration¶
seed = 1337
device = "auto"
multi_gpu = false
verbosity = "INFO"
Option |
Type |
Description |
|---|---|---|
|
int |
Required, random seed used throughout the experiment. |
|
str |
Required, execution device ( |
|
bool |
Enables multi-GPU selection logic when CUDA is available. Default: false. |
|
str |
Logging verbosity level (DEBUG, INFO, INFO:n, WARNING, ERROR, CRITICAL). Default: INFO. |
[model]¶
[model]
name = "myModel"
pretrained_path = "/path/to/model.pt"
max_ckpts = 1
ckpts_path = "/path/to/checkpoint"
Option |
Type |
Description |
|---|---|---|
|
str |
Required, model identifier used by the example factory. |
|
str |
Path to checkpoint weights. Default: none. |
|
int |
Number of retained model checkpoints after CL. Default: 0 (no checkpoint is saved). |
|
str |
Checkpoint output directory. |
[data]¶
[data]
name = "dataset"
path = "data.csv"
batch_size = 32
Option |
Type |
Description |
|---|---|---|
|
str |
Required, dataset/example identifier. |
|
str |
Reguired, dataset location. |
|
int |
Streaming batch size. Default 1 (inference/drift detection is done on each individual sample). |
[train]¶
[train]
batch_size = 64
num_workers = 4
init_lr = 0.001
grad_accumulation_steps = 1
max_iter = 600
Option |
Type |
Description |
|---|---|---|
|
int |
Required, taining batch size. |
|
int |
Required, dataLoader workers. |
|
float |
Required, initial learning rate. |
|
int |
Gradient accumulation count. Default: 1. |
|
int |
Maximum CL iterations. Default: 600. |
[continual_learning]¶
[continual_learning]
update_mode = "base"
jvp_rho_theta = 0.05
jvp_rho_x = 1.0
jvp_data_sign = 1.0
ewc_lambda = 1000.0
ewc_ema_decay = 0.95
kfac_lambda = 0.01
kfac_ema_decay = 0.95
Option |
Type |
Description |
|---|---|---|
|
str |
Required, CL strategy to use (base, jvp_reg, ewc_online, kfac_online, none). The |
|
float |
SAM parameter-perturbation radius ( |
|
float |
Perturbation radius applied to the historical inputs along the drift direction. |
|
float |
Sign of the historical-input perturbation: |
|
float |
EWC regularization strength ( |
|
float |
EMA decay for online Fisher prior in EWC. Default: 0.95. |
|
float |
KFAC penalty strength ( |
|
float |
EMA decay for running Kronecker factors in KFAC mode. Default: 0.95. |
Details about the CL algorithms available can be found in docs/continuous_learning.md
[drift_detection]¶
All the parameters are optional. Default values are provided in the examples below.
Core settings:
[drift_detection]
detector_name = "ADWINDetector"
detection_interval = 10
aggregation = "mean"
metric_index = 0
reset_after_learning = false
max_stream_updates = 20
Option |
Type |
Description |
|---|---|---|
|
str |
Drift detection algorithm (ADWINDetector, KSWINDetector, PageHinkleyDetector, ModelPerformanceDetector, ModelEvalDetector, EnsembleDetector). |
|
int |
Check drift every N monitored batches. If |
|
str |
How buffered metric values are aggregated before detector update. Supported by monitor: |
|
int |
Index into |
|
bool |
If true, detector state resets after each CL event. |
|
int |
Monitoring stops after this many stream extensions. |
ADWIN:
adwin_delta = 0.002
adwin_minor_threshold = 0.3
adwin_moderate_threshold = 0.6
Option |
Type |
Description |
|---|---|---|
|
float |
ADWIN confidence/sensitivity parameter. |
|
float |
ADWIN regime threshold (CL boundary). |
|
float |
ADWIN regime threshold (fine-tuning boundary). |
KSWIN:
kswin_alpha = 0.005
kswin_window_size = 100
kswin_stat_size = 30
kswin_seed = 1337
Option |
Type |
Description |
|---|---|---|
|
float |
KSWIN significance level. |
|
int |
KSWIN reference window size. |
|
int |
KSWIN recent sample window size. |
|
int |
Seed for KSWIN’s reference window sampling. Optional; leave unset for the historical non-reproducible behaviour. |
Page-Hinkley:
ph_min_instances = 30
ph_delta = 0.005
ph_threshold = 50
ph_alpha = 0.9999
Option |
Type |
Description |
|---|---|---|
|
int |
Page-Hinkley warm-up count before detection is meaningful. |
|
float |
Page-Hinkley change magnitude parameter. |
|
float |
Page-Hinkley trigger threshold. |
|
float |
Page-Hinkley forgetting factor. |
Ensemble:
detector_name = "EnsembleDetector"
ensemble_detectors = ["ADWINDetector", "KSWINDetector", "PageHinkleyDetector"]
ensemble_voting = "majority"
Option |
Type |
Description |
|---|---|---|
|
list[str] |
Sub-detectors to combine. Each is built from this same |
|
str |
How sub-detector verdicts combine: |
Details about the drift detection algorithms available can be found in docs/drift_detectors.md
[visualization]¶
The visualization configuration options are optional and are used to store the results of the metrics captured during the run.
[visualization]
input = "output/results.csv"
Option |
Type |
Description |
|---|---|---|
|
str |
Path to the CSV output file storing the metrics. Default: output/output.csv. |
Metrics used:
cl/cperf_detector_flop
cl/cperf_detector_flops
cl/cperf_detector_time
cl/cperf_infer_flop
cl/cperf_infer_flops
cl/cperf_infer_time
cl/cperf_optimizer_flop
cl/cperf_optimizer_flops
cl/cperf_optimizer_time
cl/cperf_update_fwd_bwd_flop
cl/cperf_update_fwd_bwd_flops
cl/cperf_update_fwd_bwd_time
cl/drift_event_id
cl/jvp_reg_forgetting_loss
cl/jvp_reg_generation_loss
cl/jvp_reg_total_loss
cl/step
drift/confidence
drift/cperf_detector_flop
drift/cperf_detector_flops
drift/cperf_detector_time
drift/cperf_infer_flop
drift/cperf_infer_flops
drift/cperf_infer_time
drift/cperf_optimizer_flop
drift/cperf_optimizer_flops
drift/cperf_optimizer_time
drift/cperf_update_fwd_bwd_flop
drift/cperf_update_fwd_bwd_flops
drift/cperf_update_fwd_bwd_time
drift/detected
drift/metric_0
drift/regime
drift/score
drift/step
eval/accuracy
eval/bwt
eval/fwt
eval/loss
eval/step
eval/test_curr_acc
eval/test_hist_acc
eval/test_pre_cl_acc
Example output file:
step,metric,value
10,eval/accuracy,62.5
10,eval/loss,2.0406203269958496
10,eval/step,10
10,drift/score,0.0
10,drift/regime,stable
10,drift/confidence,0.998
10,drift/metric_0,44.6875
10,drift/cperf_infer_flop,1624834048.0
10,drift/cperf_infer_time,0.00556622925796546
10,drift/cperf_infer_flops,291909293113.4319
10,drift/cperf_detector_flop,0.0
10,drift/cperf_detector_time,8.56249826028943e-05
10,drift/cperf_detector_flops,0.0
10,cl/jvp_reg_total_loss,3.4211268424987793
10,cl/jvp_reg_forgetting_loss,0.0
10,cl/jvp_reg_generation_loss,3.4211268424987793
[logging]¶
All the parameters are optional.
[logging]
backend = "wandb"
experiment_name = "experiment"
mlflow_tracking_uri = "http://localhost:5000"
Option |
Type |
Description |
|---|---|---|
|
str |
Logging backend (wandb, mlflow, or none). Default: wandb. |
|
str |
W&B project name or MLflow experiment name. Default: None. |
|
str |
MLflow tracking server URI. Default: None. |
Complete Valid Example¶
seed = 1337
device = "auto"
multi_gpu = false
verbosity = "INFO"
[model]
name = "mnist"
pretrained_path = "examples/mnist/mnist.pth"
max_ckpts = 0
ckpts_path = "output/mnist"
[data]
name = "mnist"
path = ""
batch_size = 32
[train]
batch_size = 64
num_workers = 4
init_lr = 0.001
[continual_learning]
update_mode = "base"
[drift_detection]
detector_name = "ADWINDetector"
[logging]
backend = "none"
[visualization]
input = "output/results.csv"