cirron test
Run comprehensive tests for ML projects: environment validation, model testing, data pipeline verification, inference, deployed endpoint checks, and end-to-end pipelines.Usage
Options
| Option | Description | Default |
|---|---|---|
--env | Test environment setup (Python, CUDA, dependencies) | false |
--build | Test Docker container build process | false |
--requirements | Validate Python requirements.txt | false |
--unit | Run unit tests (pytest/unittest) | false |
--lint | Code quality checks (flake8/pylint) | false |
--model | Test model loading and instantiation | false |
--data | Test data loading functionality | false |
--inference | Test model inference pipeline | false |
-v, --val | Run validation tests on model accuracy | false |
-p, --path <path> | Path to validation data (used with --val) | auto-detect |
-e, --endpoint <url> | Test deployed endpoint for performance | - |
--pipeline | End-to-end ML pipeline testing | false |
-w, --watch | Watch mode for continuous testing | false |
--json | Output results in JSON format | false |
--strict | Fail fast on any errors (useful for CI) | false |
-i, --interactive | Smart test selection with presets | false |
--env, --requirements, --unit, --model, and --data.
Test Types
| Test | Flag | What it checks |
|---|---|---|
| Environment | --env | Python version (vs. pythonVersion in cirron.yaml), CUDA availability for gpuRequired: true, framework-specific GPU checks, virtual env detection |
| Build | --build | Dockerfile exists, Docker build succeeds on a test image, then cleans up |
| Requirements | --requirements | requirements.txt exists, no dependency conflicts (pip check), dry-run install |
| Unit | --unit | Discovers and runs pytest (preferred) or unittest in tests/ |
| Lint | --lint | Runs flake8 (preferred) or pylint; validates style |
| Model | --model | src/model.py exists, create_model() imports and runs, model exposes fit / predict / forward |
| Data | --data | src/data_loader.py exists, sample data available in data/sample/, loads under .cirronignore filtering |
| Inference | --inference | src/inference.py exists, ModelInference instantiates, loads models/model.joblib, predicts with real or dummy data |
| Validation | --val [-p PATH] | Accuracy/MSE/MAE on validation data, inference latency and throughput; CSV files, auto-detects common paths |
| Endpoint | --endpoint <url> | Endpoint availability, latency, throughput, success rate (10 requests by default), timeout handling |
| Pipeline | --pipeline | Runs environment → data loading → model creation → inference → validation in sequence with per-step timing |
framework in cirron.yaml:
- PyTorch:
torch.cuda.is_available()when GPU is required, forward pass with dummy data, model structure and methods. - TensorFlow: GPU device availability, prediction with dummy data, model interface.
- Scikit-Learn: presence of
fit/predict, model interface.
Examples
Configuration
The CLI detects tests from your project structure (src/model.py with create_model(), src/data_loader.py, src/inference.py with ModelInference, tests/, requirements.txt, Dockerfile) and from cirron.yaml:
-p flag → cirron.yaml config → common paths (data/validation/, data/val/, data/test/, data/sample/). Honors .cirronignore. If no trained model exists, the inference test will automatically train via train.py (Trainer class) and save to models/model.joblib.
Test Output
Watch Mode
--watch watches src/**/*.py, tests/**/*.py, and cirron.yaml, automatically re-running unit, model, data, and lint tests on change.
Test Dependencies
Auto-detected: pytest (preferred) or unittest, flake8 (preferred) or pylint, Docker, pip, requests, pandas.Recommended Project Structure
Troubleshooting
| Error / Symptom | Resolution |
|---|---|
| Model file not found | Ensure src/model.py with create_model() exists |
| Data loader file not found | Create src/data_loader.py |
| Inference file not found | Create src/inference.py with ModelInference class |
| Tests directory not found | Create tests/ directory with test files |
| Python version check failed | Ensure local Python matches pythonVersion in cirron.yaml |
CUDA not available but required | Run nvidia-smi; verify gpuRequired in cirron.yaml |
Docker build failed | Ensure Docker daemon is running |
No linter found | pip install flake8 (or pylint) |
No validation data found | Pass -p or configure test.dataPaths in cirron.yaml |
Validation data is empty | Verify CSV content and .cirronignore filtering |