ci.epochs / ci.batches - Cirron Documentation

ci.epochs() and ci.batches() are transparent iterators. Each iteration opens an auto-indexed scope around the inner iterable’s yielded value. ci.epochs(range(20)) yields exactly 0..19 while producing 20 epoch spans. Use these for custom PyTorch loops where framework hooks can’t detect boundaries (generator-based iteration, custom samplers, manual step counters). Keras and HuggingFace Trainer users don’t need them; the hooks cover epoch and step boundaries automatically.

Signatures

def epochs(iterable: Iterable[T]) -> Iterator[T]
def batches(iterable: Iterable[T]) -> Iterator[T]

Both accept any iterable and pass values through unchanged.

`ci.epochs`

Opens an auto-indexed epoch scope per iteration. Index starts at 0 and increments per iteration.

for epoch in ci.epochs(range(20)):
    train_one_epoch(epoch)

Equivalent to:

for i, epoch in enumerate(range(20)):
    with ci.scope("epoch", index=i):
        train_one_epoch(epoch)

`ci.batches`

Opens an auto-indexed batch scope per iteration. When the iterable is a torch.utils.data.DataLoader, ci.batches additionally measures stall time (time spent inside __next__, waiting for data) versus compute time (time between __next__ calls, spent on the model) and stores it as a data_load_ns attribute on the batch scope.

for batch in ci.batches(loader):
    loss = train_step(batch)

Works on any iterable, but DataLoader stall measurement is PyTorch-only.

Combined

for epoch in ci.epochs(range(20)):
    for batch in ci.batches(loader):
        loss = train_step(batch)
        ci.mark("loss", loss.item())

Pass-through semantics

Both wrappers yield whatever the inner iterable yields. Neither changes types, indexing, or the number of items. A break out of the loop closes the current scope cleanly.

Overhead

~10 μs per iteration.

ci.scope

The underlying primitive. Use directly when the wrappers don’t fit.

ci.profile

Framework hooks that cover epoch / step detection automatically.

​Signatures

​ci.epochs

​ci.batches

​Combined

​Pass-through semantics

​Overhead

​Related