Profilers

FLOP and wall-time measurement built on PyTorch’s FlopCounterMode. See FLOPS Profiler for usage.

Compute performance using PyTorch FlopCounterMode with timing.

FlopCounterMode Limitations & Coverage:

COVERED OPERATIONS (with FLOP counting): - Linear layers (nn.Linear, F.linear): matmul + bias - Convolution (nn.Conv1d/2d/3d): conv operations - Matrix multiplication (torch.matmul, @, torch.mm, torch.bmm) - Element-wise ops: add, sub, mul, div (count as FLOPs) - Activation functions: ReLU, GELU, SiLU, etc. (counted as element-wise) - Normalization: LayerNorm, BatchNorm (counted) - Loss functions: CrossEntropyLoss, MSELoss (counted as element-wise + reductions)

NOT COVERED or LIMITATIONS: - Indexing, slicing, reshape, transpose (NO FLOPs, just memory ops) - Custom autograd Functions - need explicit FLOP annotations - Some advanced operations - check flop_counter.py for full list Reference: https://github.com/pytorch/pytorch/blob/main/torch/utils/flop_counter.py

  • Relying on the accuracy of torch’s flop_counter for general use.

  • Optimizers require FLOP estimation as flop_counter does not automatically count all non-module operations.

  • For profiling the optimizer step, user must measure_flops_optimizer(tag=”opt”, model=model, device=cfg.device).

class apeiron.profilers.count_flops.FLOPSProfiler[source]

Bases: object

Profiler for measuring FLOPs and execution time of PyTorch operations.

This class uses PyTorch’s FlopCounterMode to automatically count FLOPs for most operations. For optimizer steps, use measure_flops_optimizer() which employs torch.profiler for accurate FLOP estimation.

warmup_iters

Number of warmup iterations before profiling

profiles

Dictionary storing FLOP and time measurements per tag

tag

Current measurement tag (set during active profiling)

start_time

Start time of current measurement

flop_counter

FlopCounterMode instance for current measurement

__init__(warmup_iters=1)[source]

Initialize the FLOPSProfiler.

Parameters:

warmup_iters (int) – Number of warmup iterations (default: 1)

Return type:

None

measure_flops(tag='default')[source]

Context manager for measuring FLOPs and time for a code block.

Uses FlopCounterMode to automatically count FLOPs for supported operations. Best for forward/backward passes of neural network modules.

Parameters:

tag (str) – Identifier for this measurement session (default: “default”)

Yields:

self – The profiler instance

Return type:

Generator[FLOPSProfiler, None, None]

Example

>>> profiler = FLOPSProfiler()
>>> with profiler.measure_flops("forward"):
...     output = model(input)
measure_flops_optimizer(model, device, tag='optimizer')[source]

Context manager for measuring FLOPs and time for optimizer step.

Uses torch.profiler to estimate FLOPs for optimizer operations, which are not captured by FlopCounterMode. Estimates total FLOPs by multiplying per-element operations by the number of trainable parameters.

Parameters:
  • model (torch.nn.Module) – The model being optimized (used to count parameters)

  • device (str) – Device type (‘cuda’ or ‘cpu’)

  • tag (str) – Identifier for this measurement session (default: “optimizer”)

Yields:

self – The profiler instance

Return type:

Generator[FLOPSProfiler, None, None]

Example

>>> profiler = FLOPSProfiler()
>>> with profiler.measure_flops_optimizer(model, "cuda", "optimizer"):
...     optimizer.step()
get_performance()[source]

Calculate average performance metrics across all measurements.

Computes the mean FLOPs, time, and throughput (FLOP/s) for each tag.

Returns:

  • {tag}_flop: Average FLOPs per operation

  • {tag}_time: Average time per operation (seconds)

  • {tag}_flops: Average throughput (FLOP/s)

Return type:

Dictionary containing averaged metrics with keys

print_performance(logger=None, level=0)[source]

Pretty print the performance metrics (averaged per update).

Displays a formatted table showing FLOPs, time, and throughput for each profiled operation tag, along with totals.

Parameters:
  • logger – Optional logger instance. If provided, uses logger.info(). If None, falls back to print() for backward compatibility.

  • level (int) – Verbosity level for logging (default: 0). Only used when logger is provided. Higher values are more verbose (e.g., level=1 for INFO:1, level=2 for INFO:2).

Return type:

None