cirron test

Run comprehensive tests for ML projects: environment validation, model testing, data pipeline verification, inference, deployed endpoint checks, and end-to-end pipelines.

Usage

cirron test [options]

Options

Option	Description	Default
`--env`	Test environment setup (Python, CUDA, dependencies)	`false`
`--build`	Test Docker container build process	`false`
`--requirements`	Validate Python `requirements.txt`	`false`
`--unit`	Run unit tests (pytest/unittest)	`false`
`--lint`	Code quality checks (flake8/pylint)	`false`
`--model`	Test model loading and instantiation	`false`
`--data`	Test data loading functionality	`false`
`--inference`	Test model inference pipeline	`false`
`-v, --val`	Run validation tests on model accuracy	`false`
`-p, --path <path>`	Path to validation data (used with `--val`)	auto-detect
`-e, --endpoint <url>`	Test deployed endpoint for performance	-
`--pipeline`	End-to-end ML pipeline testing	`false`
`-w, --watch`	Watch mode for continuous testing	`false`
`--json`	Output results in JSON format	`false`
`--strict`	Fail fast on any errors (useful for CI)	`false`
`-i, --interactive`	Smart test selection with presets	`false`

When no specific test flags are passed, the default suite runs --env, --requirements, --unit, --model, and --data.

Test Types

Test	Flag	What it checks
Environment	`--env`	Python version (vs. `pythonVersion` in `cirron.yaml`), CUDA availability for `gpuRequired: true`, framework-specific GPU checks, virtual env detection
Build	`--build`	`Dockerfile` exists, Docker build succeeds on a test image, then cleans up
Requirements	`--requirements`	`requirements.txt` exists, no dependency conflicts (`pip check`), dry-run install
Unit	`--unit`	Discovers and runs pytest (preferred) or unittest in `tests/`
Lint	`--lint`	Runs flake8 (preferred) or pylint; validates style
Model	`--model`	`src/model.py` exists, `create_model()` imports and runs, model exposes `fit` / `predict` / `forward`
Data	`--data`	`src/data_loader.py` exists, sample data available in `data/sample/`, loads under `.cirronignore` filtering
Inference	`--inference`	`src/inference.py` exists, `ModelInference` instantiates, loads `models/model.joblib`, predicts with real or dummy data
Validation	`--val [-p PATH]`	Accuracy/MSE/MAE on validation data, inference latency and throughput; CSV files, auto-detects common paths
Endpoint	`--endpoint <url>`	Endpoint availability, latency, throughput, success rate (10 requests by default), timeout handling
Pipeline	`--pipeline`	Runs environment → data loading → model creation → inference → validation in sequence with per-step timing

Framework-specific behavior is applied automatically based on framework in cirron.yaml:

PyTorch: torch.cuda.is_available() when GPU is required, forward pass with dummy data, model structure and methods.
TensorFlow: GPU device availability, prediction with dummy data, model interface.
Scikit-Learn: presence of fit / predict, model interface.

Examples

# Defaults (env, requirements, unit, model, data)
cirron test

# Specific components
cirron test --model
cirron test --unit --lint

# Build, inference, validation
cirron test --build
cirron test --inference
cirron test --val -p data/test/

# Deployed endpoint
cirron test --endpoint https://api.example.com/predict

# End-to-end pipeline
cirron test --pipeline

# Watch mode for development
cirron test --watch

# CI mode — fail fast, JSON output
cirron test --strict --json

Configuration

The CLI detects tests from your project structure (src/model.py with create_model(), src/data_loader.py, src/inference.py with ModelInference, tests/, requirements.txt, Dockerfile) and from cirron.yaml:

{
  "name": "my-model",
  "version": "1.0.0",
  "framework": "pytorch",
  "pythonVersion": "3.9",
  "gpuRequired": false,
  "test": {
    "dataPaths": {
      "validation": "data/validation/",
      "sample": "data/sample/"
    },
    "fallbackToDummy": true
  }
}

Validation data path resolution: -p flag → cirron.yaml config → common paths (data/validation/, data/val/, data/test/, data/sample/). Honors .cirronignore. If no trained model exists, the inference test will automatically train via train.py (Trainer class) and save to models/model.joblib.

Test Output

🧪 Test Results

✓ Environment tests
✓ Requirements tests
✓ Unit tests
✓ Model tests
✓ Data tests

All 5 test suites passed!

🧪 Test Results

✓ Environment tests
✗ Requirements tests
   requirements.txt not found
✓ Unit tests
✗ Model tests
   Model file not found
✓ Data tests

2 of 5 test suites failed

Tests run sequentially with per-test error handling. Failures don’t stop the suite.

Watch Mode

--watch watches src/**/*.py, tests/**/*.py, and cirron.yaml, automatically re-running unit, model, data, and lint tests on change.

Test Dependencies

Auto-detected: pytest (preferred) or unittest, flake8 (preferred) or pylint, Docker, pip, requests, pandas.

Recommended Project Structure

my-project/
├── src/
│   ├── model.py          # Required: create_model()
│   ├── data_loader.py    # Required for --data
│   ├── inference.py      # Required: ModelInference class
│   └── train.py          # Required: Trainer class
├── tests/                # Unit tests
├── data/
│   ├── sample/           # Sample data for testing
│   └── validation/       # Validation data
├── models/               # Trained models
├── requirements.txt      # Required for --requirements
├── Dockerfile            # Required for --build
└── cirron.yaml           # Project configuration

Troubleshooting

Error / Symptom	Resolution
Model file not found	Ensure `src/model.py` with `create_model()` exists
Data loader file not found	Create `src/data_loader.py`
Inference file not found	Create `src/inference.py` with `ModelInference` class
Tests directory not found	Create `tests/` directory with test files
Python version check failed	Ensure local Python matches `pythonVersion` in `cirron.yaml`
`CUDA not available but required`	Run `nvidia-smi`; verify `gpuRequired` in `cirron.yaml`
`Docker build failed`	Ensure Docker daemon is running
`No linter found`	`pip install flake8` (or `pylint`)
`No validation data found`	Pass `-p` or configure `test.dataPaths` in `cirron.yaml`
`Validation data is empty`	Verify CSV content and `.cirronignore` filtering

CI/CD

# Full suite in CI
cirron test --strict

# Pre-deployment
cirron test --env --build --model --inference
cirron test --val -p data/validation/
cirron test --endpoint https://production-api.com/predict

Cirron CLI

Commands

Utils

Templates

test

cirron test

Usage

Options

Test Types

Examples

Configuration

Test Output

Watch Mode

Test Dependencies

Recommended Project Structure

Troubleshooting

CI/CD

​cirron test

​Usage

​Options

​Test Types

​Examples

​Configuration

​Test Output

​Watch Mode

​Test Dependencies

​Recommended Project Structure

​Troubleshooting

​CI/CD

cirron test

Usage

Options

Test Types

Examples

Configuration

Test Output

Watch Mode

Test Dependencies

Recommended Project Structure

Troubleshooting

CI/CD