Test Inventory - TorchCTS

Focused By Design

This inventory is organized around backend-relevant PyTorch behavior. It combines PyTorch OpInfo-sourced tests, generated dispatcher cases, hand-authored backend semantics, harness self-checks, and targeted path-shape cases.

It is not intended to reproduce every framework, frontend, distributed, packaging, or internal implementation test in the full PyTorch project.

The inventory is small enough to support repeated backend development and CI runs. Coverage audits keep the smaller scope accountable by showing covered, pending, excluded, and unknown dispatcher surfaces.

Current Shape

The July 20, 2026 stats run collected 19,395 pytest nodes at semantic level 8.

18,578Executable nodes

814Structured filtered nodes

3Pytest skip-marked nodes

19,222Parameterized node IDs

173Unparameterized node IDs

Test Breakdown

PyTorch OpInfo-derived operator tests means TorchCTS builds tests from PyTorch's own OpInfo operator metadata and sample inputs. That gives TorchCTS broad operator coverage without pretending those cases were hand-written one by one.

13,070PyTorch OpInfo-derived operator tests

4,258Generated dispatcher and variant tests

2,053Hand-authored backend behavior tests

14Harness self-checks

PyTorch OpInfo-derived operator tests13,070 nodes: forward 10,553, backward 2,315, errors 202.

Generated4,258 nodes: foreach/fused 1,969, out 898, functional 578, other 813.

Hand-authored2,053 nodes: path-shape 850, operators other 478, compiler 144, dtypes 135, workloads other 120, other 326.

Harness self-checks14 nodes.

Source site-stats.md Generated 2026-07-20 at 21:07 UTC. Small leaf families are grouped so the chart stays readable.

Suite Counts

Top-level test kinds: PyTorch OpInfo-derived operator tests 13,070, generated 4,258, hand-authored 2,053, and 14 harness self-checks.

Suite	Collected Nodes
PyTorch OpInfo-derived operator tests	13,070
generated	4,258
operators	1,193
compiler	144
dtypes	135
workloads	255
strides	87
autograd	78
training	56
stress	30
rng	28
memory	24
Harness self-checks	14
device_api	12
errors	5
multi_device	3
serialization	3

Generated File Counts

Generated Test File	Collected Nodes
test_foreach_fused.py	1,969
test_out_variants.py	898
test_functional_variants.py	578
test_inplace_variants.py	346
test_oracle_surfaces.py	273
test_view_aliases.py	100
test_autograd_backward_variants.py	38
test_layout_storage_variants.py	24
test_factories.py	20
test_rng_variants.py	12

Path-shape tests are tracked separately because they come from a curated corpus instead of the generated dispatcher suite.

Path-Shape Test File	Collected Nodes
torchcts/operators/test_operator_path_shapes.py	715
torchcts/workloads/test_workload_path_shapes.py	135

Semantic Level Inventory

Semantic levels describe run depth. Higher levels do not replace lower levels. They add more behavior, more resource pressure, or more backend integration surface.

Level 1

Core primitives for continuous backend validation: basic tensor creation, simple unary and binary operations, basic dtype behavior, and the smallest generated dispatcher cases.

Level 2

Normal operator correctness: broad PyTorch OpInfo-derived forward and backward cases, common tensor-producing operators, common tensor-consuming operators, and expected error behavior.

Level 3

Mainstream framework semantics: mutation, aliasing, out= behavior, view behavior, RNG behavior, metadata behavior, and generated functional/inplace/out variants.

Level 4

Production bring-up depth: training-adjacent coverage, autograd-adjacent behavior, backend-family cases, and broader generated variant coverage that starts to expose real backend gaps.

Level 5

Advanced tensor behavior: numeric edge cases, layout and storage behavior, sparse and nested coverage, noncontiguous tensors, channels-last paths, stride-sensitive cases, and most standard-tier path-shape rows.

Level 6

Backend integration: compiler paths, device API, streams and events, allocator behavior, guarded allocation, quantization-adjacent plumbing, and low-level implementation surfaces.

Level 7

Heavy integration: realistic workloads, model-shaped tests, multi-device behavior, and selected path-shape rows that are more expensive or more dependent on the target hardware.

Level 8

Release-depth validation: stress, adversarial behavior, large tensor pressure, rapid allocation patterns, edge numerics, and cases intended for final validation rather than every local edit.

Level	Nodes	Executable	Filtered?Structured filtered nodes are TorchCTS selection decisions recorded before pytest execution where possible. They are not pytest skips and they are not passes.	Skipped?Pytest skip-marked nodes were collected by pytest and marked skipped for runtime conditions such as CPU-only applicability or device-count requirements.	Surfaces	Gen'd
1	436	435	1	0	418	418
2	13,789	13,655	134	0	964	143
3	1,085	741	341	3	1,049	711
4	2,405	2,174	231	0	382	273
5	1,248	1,196	52	0	376	343
6	199	171	28	0	19	19
7	203	179	24	0	6	0
8	30	27	3	0	0	0

Levels 7 and 8 live primarily in hand-authored workload, multi-device, and stress tests, so generated-dispatcher counts can be zero while pytest nodes are nonzero.

Path-Shape Corpus

The path-shape corpus tracks reviewed cases for shapes, layouts, strides, masks, model roles, dtypes, resource tiers, and cost classes. It is built to hit backend branch points, not to enumerate every possible tensor size.

1,320Tracked path-shape rows

850Default rows

470Heavy rows

12Families

By corpus level, TorchCTS tracks 890 level 5 rows, 80 level 7 rows, and 350 level 8 rows. The default pytest collection currently includes 770 level 5 rows and 80 level 7 rows.

Family	Total Rows	Default Rows
matmul	210	140
attention	150	80
convolution	140	90
reduction	120	85
indexing	110	80
spatial	100	65
model_patterns	95	55
normalization	90	65
fft	85	45
sorting	80	55
broadcasting	75	55
linear_algebra	65	35