> ## Documentation Index > Fetch the complete documentation index at: https://docs.cirron.com/llms.txt > Use this file to discover all available pages before exploring further. # test > Run tests for ML projects locally and in CI. # cirron test Run comprehensive tests for ML projects: environment validation, model testing, data pipeline verification, inference, deployed endpoint checks, and end-to-end pipelines. ## Usage ```bash theme={null} cirron test [options] ``` ## Options | Option | Description | Default | | ---------------------- | --------------------------------------------------- | ----------- | | `--env` | Test environment setup (Python, CUDA, dependencies) | `false` | | `--build` | Test Docker container build process | `false` | | `--requirements` | Validate Python `requirements.txt` | `false` | | `--unit` | Run unit tests (pytest/unittest) | `false` | | `--lint` | Code quality checks (flake8/pylint) | `false` | | `--model` | Test model loading and instantiation | `false` | | `--data` | Test data loading functionality | `false` | | `--inference` | Test model inference pipeline | `false` | | `-v, --val` | Run validation tests on model accuracy | `false` | | `-p, --path ` | Path to validation data (used with `--val`) | auto-detect | | `-e, --endpoint ` | Test deployed endpoint for performance | - | | `--pipeline` | End-to-end ML pipeline testing | `false` | | `-w, --watch` | Watch mode for continuous testing | `false` | | `--json` | Output results in JSON format | `false` | | `--strict` | Fail fast on any errors (useful for CI) | `false` | | `-i, --interactive` | Smart test selection with presets | `false` | When no specific test flags are passed, the default suite runs `--env`, `--requirements`, `--unit`, `--model`, and `--data`. ## Test Types | Test | Flag | What it checks | | ------------ | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | Environment | `--env` | Python version (vs. `pythonVersion` in `cirron.yaml`), CUDA availability for `gpuRequired: true`, framework-specific GPU checks, virtual env detection | | Build | `--build` | `Dockerfile` exists, Docker build succeeds on a test image, then cleans up | | Requirements | `--requirements` | `requirements.txt` exists, no dependency conflicts (`pip check`), dry-run install | | Unit | `--unit` | Discovers and runs pytest (preferred) or unittest in `tests/` | | Lint | `--lint` | Runs flake8 (preferred) or pylint; validates style | | Model | `--model` | `src/model.py` exists, `create_model()` imports and runs, model exposes `fit` / `predict` / `forward` | | Data | `--data` | `src/data_loader.py` exists, sample data available in `data/sample/`, loads under `.cirronignore` filtering | | Inference | `--inference` | `src/inference.py` exists, `ModelInference` instantiates, loads `models/model.joblib`, predicts with real or dummy data | | Validation | `--val [-p PATH]` | Accuracy/MSE/MAE on validation data, inference latency and throughput; CSV files, auto-detects common paths | | Endpoint | `--endpoint ` | Endpoint availability, latency, throughput, success rate (10 requests by default), timeout handling | | Pipeline | `--pipeline` | Runs environment → data loading → model creation → inference → validation in sequence with per-step timing | Framework-specific behavior is applied automatically based on `framework` in `cirron.yaml`: * **PyTorch**: `torch.cuda.is_available()` when GPU is required, forward pass with dummy data, model structure and methods. * **TensorFlow**: GPU device availability, prediction with dummy data, model interface. * **Scikit-Learn**: presence of `fit` / `predict`, model interface. ## Examples ```bash theme={null} # Defaults (env, requirements, unit, model, data) cirron test # Specific components cirron test --model cirron test --unit --lint # Build, inference, validation cirron test --build cirron test --inference cirron test --val -p data/test/ # Deployed endpoint cirron test --endpoint https://api.example.com/predict # End-to-end pipeline cirron test --pipeline # Watch mode for development cirron test --watch # CI mode — fail fast, JSON output cirron test --strict --json ``` ## Configuration The CLI detects tests from your project structure (`src/model.py` with `create_model()`, `src/data_loader.py`, `src/inference.py` with `ModelInference`, `tests/`, `requirements.txt`, `Dockerfile`) and from `cirron.yaml`: ```json theme={null} { "name": "my-model", "version": "1.0.0", "framework": "pytorch", "pythonVersion": "3.9", "gpuRequired": false, "test": { "dataPaths": { "validation": "data/validation/", "sample": "data/sample/" }, "fallbackToDummy": true } } ``` Validation data path resolution: `-p` flag → `cirron.yaml` config → common paths (`data/validation/`, `data/val/`, `data/test/`, `data/sample/`). Honors `.cirronignore`. If no trained model exists, the inference test will automatically train via `train.py` (`Trainer` class) and save to `models/model.joblib`. ## Test Output ``` 🧪 Test Results ✓ Environment tests ✓ Requirements tests ✓ Unit tests ✓ Model tests ✓ Data tests All 5 test suites passed! ``` ``` 🧪 Test Results ✓ Environment tests ✗ Requirements tests requirements.txt not found ✓ Unit tests ✗ Model tests Model file not found ✓ Data tests 2 of 5 test suites failed ``` Tests run sequentially with per-test error handling. Failures don't stop the suite. ## Watch Mode `--watch` watches `src/**/*.py`, `tests/**/*.py`, and `cirron.yaml`, automatically re-running unit, model, data, and lint tests on change. ## Test Dependencies Auto-detected: pytest (preferred) or unittest, flake8 (preferred) or pylint, Docker, pip, requests, pandas. ## Recommended Project Structure ``` my-project/ ├── src/ │ ├── model.py # Required: create_model() │ ├── data_loader.py # Required for --data │ ├── inference.py # Required: ModelInference class │ └── train.py # Required: Trainer class ├── tests/ # Unit tests ├── data/ │ ├── sample/ # Sample data for testing │ └── validation/ # Validation data ├── models/ # Trained models ├── requirements.txt # Required for --requirements ├── Dockerfile # Required for --build └── cirron.yaml # Project configuration ``` ## Troubleshooting | Error / Symptom | Resolution | | --------------------------------- | ------------------------------------------------------------ | | Model file not found | Ensure `src/model.py` with `create_model()` exists | | Data loader file not found | Create `src/data_loader.py` | | Inference file not found | Create `src/inference.py` with `ModelInference` class | | Tests directory not found | Create `tests/` directory with test files | | Python version check failed | Ensure local Python matches `pythonVersion` in `cirron.yaml` | | `CUDA not available but required` | Run `nvidia-smi`; verify `gpuRequired` in `cirron.yaml` | | `Docker build failed` | Ensure Docker daemon is running | | `No linter found` | `pip install flake8` (or `pylint`) | | `No validation data found` | Pass `-p` or configure `test.dataPaths` in `cirron.yaml` | | `Validation data is empty` | Verify CSV content and `.cirronignore` filtering | ## CI/CD ```bash theme={null} # Full suite in CI cirron test --strict # Pre-deployment cirron test --env --build --model --inference cirron test --val -p data/validation/ cirron test --endpoint https://production-api.com/predict ```