Reference
Use this page when you need exact names, flags, fields, or file locations.
Reference Index
CLI Reference
Commands for init, run, show-skips, report, coverage, canonical backend evidence, manifest checks, cache refresh for PyTorch OpInfo-derived tests, and MPS triage.
Manifest Reference
Field names, accepted values, capabilities, dtype declarations, hardware settings, resource limits, and validation rules.
Test Inventory
Current suite shape, semantic levels, generated suite counts, dtype tokens, and collection stats.
Path-Shape Corpus
Curated shape, layout, stride, resource-tier, cost-class, model-role, and dtype-group cases for targeted backend branch coverage.
Backends
In-tree backend families, install variant selection, runtime device selection, backend-pack gates, and PrivateUse1 backend examples.
Artifacts
What to save from a run and which files matter for debugging, release gates, and maintainer review.
Artifact Locations
results/*_latest.jsonActive-run recovery snapshot or completed-run reference to the canonical history artifact.
results/*_history/Canonical completed result artifacts. Large runs use compact JSON storage when it reduces artifact size. TorchCTS tools load that format as normal result data.
results/*.mdHuman report generated from the resolved result artifact.
results/*_runlog*Rolling execution trace with the latest 32 test starts used for crash and hang diagnosis.
results/coverage/audit.jsonDispatcher coverage audit.
results/coverage/generated_cases.jsonGenerated coverage cases.
results/coverage/pending_review.jsonMachine-readable pending and excluded coverage review records.
results/*_harness_probe_failures_*.jsonlFailure-only diagnostic probe records.
results/*_opinfo_oracle_failures_*.jsonlDiagnostic records created when TorchCTS cannot build a valid CPU-based expected result for a PyTorch OpInfo-derived sample. They are not backend results or development-only oracle QA records.
evidence/backends/Canonical tracked backend evidence records for backend-gated coverage.
Evidence Storage Model
Compact operator contracts, operator metadata, the path-shape corpus, templates, and known crash rules needed by an installed TorchCTS package.
Tracked under evidence/pytorch/dtype-contracts/. Used to audit and regenerate compact runtime operator contracts.
Tracked under evidence/backends/. Records are path-free and avoid host identity, full audit snapshots, and transport archive metadata.
Tracked in source-only test, evidence, and generator directories. They validate TorchCTS-owned references but do not appear in installed backend runs.
Written under results/. Completed runs keep canonical history artifacts and a latest reference that TorchCTS commands resolve.
Committed under sample-results/ so engineers can inspect real reports and compact/reference-aware result artifacts.
Oracle QA Files
tests/oracles/cases/Reviewed fixed cases used to test TorchCTS-owned references.
evidence/oracles/Source records, manifests, and raw evidence used to review and regenerate oracle QA cases.
scripts/oracle_fixtures/Development tools that create and validate oracle QA fixtures.
Runtime reference implementations ship as code. Their QA cases, generators, and raw evidence stay in the source repository and do not contribute to backend result totals.
Package Artifacts
Wheels and sdists include the files needed at runtime. Source evidence and backend evidence stay in the TorchCTS repository.
Compact runtime operator contracts, operator metadata, the path-shape corpus, known crash isolation rules, templates, and the install planner source.
PyTorch source evidence, backend evidence, oracle QA cases, oracle generators, raw oracle evidence, sample results, local caches, generated site installer artifacts, local run output, and release-only audit scratch files.
The installed package has the data needed to run. The repository keeps the larger evidence needed to audit and regenerate that data.
Installer Behavior
The installers choose a PyTorch wheel family. They do not decide which backend is under test.
TorchCTS 0.4.1 supports PyTorch 2.7.0-2.12.1. The installer uses the bounded requirement torch>=2.7.0,<2.12.2. If an existing PyTorch install is older or newer than that range, the installer warns and points to the validated range. Broken PyTorch imports still fail.
TORCHCTS_TORCH_VARIANT
Forces the installer to use a specific PyTorch wheel family.
cpuCPU PyTorch wheel family. Runtime device hint is cpu.cudaCUDA PyTorch wheel family. Runtime device hint is cuda.rocmROCm PyTorch wheel family. PyTorch exposes ROCm devices through the cuda runtime namespace.xpuIntel XPU PyTorch wheel family. Runtime device hint is xpu.mpsmacOS default PyTorch wheel path. Runtime device hint is mps.nvidiaamdhipintelAccepted aliases for cuda, rocm, rocm, and xpu.TORCHCTS_NON_INTERACTIVE
Controls installer prompting.
1CI-safe mode. Do not prompt.unsetInteractive mode when the installer needs a choice.TORCHCTS_UPGRADE_TORCH
Controls installer behavior when the current environment is outside the validated PyTorch range.
1Install a PyTorch version in the validated torch>=2.7.0,<2.12.2 range.unsetWarn about an out-of-range existing PyTorch install without replacing it.Runtime Backend Selection
Use torchcts run --device ... to select the backend under test. The installer only chooses the PyTorch build family.
Common Questions
TorchCTS uses PyTorch resources such as OpInfo metadata, operator schemas, dispatcher behavior, and CPU reference behavior, then adds backend configuration, focused selection, generated and hand-authored backend tests, structured filtered accounting, coverage audits, and persistent artifacts. Backend teams can use TorchCTS alongside the upstream tests appropriate to their development environment.
No. PyTorch's tests validate the framework across many subsystems and environments. TorchCTS provides a smaller workflow focused on developing and validating an individual backend.
No. The focused tests are one part of the system. The manifest, operator contracts, coverage audit, filtering rules, result artifacts, reports, and crash records connect development work to conformance and release review.
Yes. Use a focused TorchCTS selection while implementing a backend behavior, rerun it as the implementation changes, then retain the appropriate broader command as regression coverage. TorchCTS supplies the tests and expected behavior, while the backend project owns the implementation workflow.
The focused development suite becomes broader regression and release coverage as the backend grows. This lets teams find unsupported paths, semantic regressions, crashes, and coverage gaps during development instead of relying on downstream users to discover them.
TorchCTS-owned references have a separate source-repository QA suite built from reviewed fixed records. It checks values, legal-output rules, backward formulas, routing, provenance, and schema behavior without adding those QA cases to backend results. See Oracle Quality Assurance.
No. Filtered means TorchCTS did not run the case and recorded why.
check-manifest?Run it before interpreting backend results. A schema or configuration problem can otherwise look like an implementation failure.
Filtered cases are TorchCTS accounting for cases removed before pytest execution, such as unconfigured dtypes or out-of-contract operator cases. Pytest skip-marked means pytest collected the node and marked it skipped for a runtime condition.
No. Run TorchCTS with isolation when needed. Isolation protects the parent process, but the crash still counts and stays in the result.
Yes. Use --device cpu for harness checks, CPU behavior work, and CPU-build evidence. CPU validation is not evidence for an accelerator backend.
Keep overrides narrow: one category, one dtype, and reviewed numeric values. Broad tolerance changes make real correctness problems easier to miss.
TorchCTS 0.4.1 validates PyTorch 2.7.0-2.12.1 and uses torch>=2.7.0,<2.12.2. Outside that range, operator contracts are not validated and the installer warns.
Sample result sets live in the TorchCTS repository at sample-results.