Span-Overflow Hint Analysis

Background

Spyre kernels must keep each core’s memory-address span within the hardware limit (MAX_SPAN_BYTES, normally 256 MiB). Normal work_division splits work across cores, but some physical layouts still expose a span that is too large for one core. When that happens, backend compilation can fail with Work Division warnings, deeptools mutable-address failures, or immediate/EAR boundary errors.

span_overflow_hint_analysis.py is the compiler-generated version of a user spyre_hint: it detects span-risky ComputedBuffer ops and asks existing coarse tiling to run those ops in smaller output-range tiles.

The planner does not mutate IR directly. It computes a SpanOverflowTilePlan. The adapter in coarse_tile.py converts that plan into the same DimHint and coarse-tile group format used by manual spyre_hint.

span_overflow_hint_analysis
  -> coarse_tile.span_overflow_groups
  -> coarse_tile
  -> CountedLoopSchedulerNode
  -> LoopSpec codegen

High-Level Contract

The planner answers this question for one op:

If Work Division gives this op no useful span reduction, can output-range coarse tiling make every relevant output/input span fit under MAX_SPAN_BYTES?

The important design choice is conservative validation: automatic tiles must be safe even if Work Division later chooses a different split dimension or cannot split the span-driving coordinate at all. Work Division may still improve parallelism, but correctness must not depend on it.

Concrete Examples

This section gives examples for the main cases the pass handles. Shapes are representative; exact split counts can vary with Spyre physical layout, dtype, SENCORES, and MAX_SPAN_BYTES.

1. Pointwise Output Span Overflow

Example operation:

def fn(x, y):
    return x + y

x.shape == y.shape == (1, 8195, 256, 64)

The output is large enough that one physical output coordinate can span more than the limit. The Pointwise path collects an output candidate and emits an output-range tile, for example:

selected_host_dim = 1
split_count       = 5

Conceptually this is equivalent to:

with spyre_hint(num_tiles_per_dim={"H": 5}):
    out = x + y

Code flow:

plan_span_overflow_tile
  -> Pointwise branch
  -> _has_indirect_reads(op) == False
  -> _output_span_candidates_from_op
  -> _search_min_cost_tile_plan
  -> _post_tile_layout_for_splits
  -> _remaining_span_candidates_after_tile
  -> SpanOverflowTilePlan(levels=(host_dim=1, split_count=5))

2. Pointwise With Indirect/Gather Read

Example pattern:

def fn(x, index):
    return x[index] + 1

Even if the resulting output layout is large, automatic span-overflow tiling is skipped because indirect access needs the dedicated indirect-access path. The planner returns None.

Code flow:

plan_span_overflow_tile
  -> Pointwise branch
  -> _has_indirect_reads(op) == True
  -> return None

3. Reduction Output Span Overflow

Example operation:

def fn(x):
    return x.sum(dim=-1)

x.shape == (1, 8195, 256, 64)
out.shape == (1, 8195, 256)

The reduction output itself can have an unsafe physical span. The planner uses the same output-candidate path as Pointwise, but through the Reduction branch. If a safe output-range tile exists, it emits a normal output-dim plan.

Code flow:

plan_span_overflow_tile
  -> Reduction branch
  -> _has_indirect_reads(op) == False
  -> op.data.ranges is not empty
  -> _output_span_candidates_from_op
  -> _input_span_candidates
  -> _search_min_cost_tile_plan

4. Reduction Input Span Controlled By Output Dim

Example operation:

def fn(x):
    return x.sum(dim=0)

x.shape   == (2, 20, 16, 64)
out.shape == (20, 16, 64)

The output may be smaller than the input, but reading x can still span too much memory. If the overflowing input physical coordinate is controlled by an output symbol, such as the output 20 dimension, output-range tiling can reduce the input span.

A small codegen test uses a patched span limit and checks for:

LoopSpec(
count=sympify('10')
op='sum'
tiled_symbols=[[sympify('c0')]]

Code flow:

plan_span_overflow_tile
  -> Reduction branch
  -> _output_span_candidates_from_op
  -> _input_span_candidates
       -> _input_span_infos_controlled_by_output_dims
          -> device coordinate contains output symbol
          -> create input-derived SpanOverflowCandidate
  -> _search_min_cost_tile_plan

5. BMM / LM-Head Input Span Controlled By Output Dim

Example operation:

def lm_head(x, weight):
    return torch.nn.functional.linear(x, weight)

x.shape      == (2, 64)
weight.shape == (1024, 64)
out.shape    == (2, 1024)

Lowering can represent this as a restickify producer followed by a batch-matmul-style reduction. The BMM symbol model is:

C[b, m, n] = reduce_k A[b, m, k] * B[b, k, n]

Spans controlled by b, m, or n are output-range tileable. Spans controlled by k are not. In the LM-head lowering, the restickify producer can be auto-tiled independently. The BMM consumer is not automatically fused with that producer yet, so current codegen coverage checks for a restickify LoopSpec plus a plain BMM consumer:

LoopSpec(
count=sympify('4')
op='ReStickifyOpHBM'
tiled_symbols=[[sympify('c0')]]

op='batchmatmul'

Code flow:

plan_span_overflow_tile
  -> Reduction branch
  -> _is_batch_matmul_reduction(op) == True
  -> _bmm_output_symbol_to_dim
       -> verify exactly one reduction-only symbol, k for input-span candidates
  -> _input_span_candidates
  -> _search_min_cost_tile_plan

6. Multiple Overflowing Host Dims

Example pattern:

x.shape == (4096, 4096, 4096, 64)
out = x + 1

More than one physical coordinate can independently exceed the span limit. The planner creates candidates for each overflowing host dim, generates legal split candidates for each dim, and searches combinations.

A multi-level result looks like:

SpanOverflowTilePlan(
    levels=(
        SpanOverflowTileLevel(selected_host_dim=0, split_count=2),
        SpanOverflowTileLevel(selected_host_dim=1, split_count=2),
    ),
)

Code flow:

_output_span_candidates_from_op -> candidates for dim 0 and dim 1
_candidate_host_dims            -> order by span pressure
_split_candidates_for_host_dim  -> legal divisors per dim
_iter_split_combos              -> cheapest combos first
_remaining_span_candidates_after_tile -> accept only if all spans are safe

7. Input Stick-Alignment Rejection

Example pattern:

output dim m is safe to split in the output layout,
but the same m symbol maps to the within-stick dim of a transposed BMM input.

The planner rejects the split if it would cut through physical sticks in any input dependency. This prevents a plan that is legal for the output tensor but unsafe for an input tensor.

_input_stick_alignment_error checks every input dimension the split symbol contributes to, not only a dimension it controls alone. A single input dimension’s coordinate can be jointly controlled by the symbol together with another symbol (e.g. an interleaved/collapsed physical stride after a view or transpose) – checking only dimensions where the symbol is the sole free symbol would silently skip stick alignment for that dimension entirely.

Both _post_tile_stick_alignment_error and _input_stick_alignment_error depend on _within_stick_host_dim to identify which host dim is the within-stick dim for a given layout. If no host stride exactly matches the device layout’s stride-map entry, _within_stick_host_dim returns None rather than guess a fallback dim, and the caller treats that as “cannot verify, reject the split” – matching the “skip/reject rather than guess” discipline used elsewhere in this pass (ambiguous BMM symbol maps return empty; _resize_device_layout raises rather than guess an ambiguous stick dim by elimination).

Code flow:

_split_candidates_for_host_dim
  -> _post_tile_stick_alignment_error(op.layout, host_dim, split)
       -> _within_stick_host_dim(op.layout)
  -> _input_stick_alignment_error(op, host_dim, split)
       -> map output host dim to output symbol
       -> find every input dim that symbol contributes to
       -> check that input layout's stick dim is not cut, via
          _post_tile_stick_alignment_error -> _within_stick_host_dim

8. Reduction-Controlled Span: Known Unsupported Case

Example operation:

def fn(x):
    return x.sum(dim=-1)

If the only overflowing input coordinate is controlled by the reduction symbol k, output-range tiling cannot fix it. This pass intentionally skips that candidate.

Code flow:

_input_span_infos_controlled_by_output_dims
  -> coordinate free symbols include k
  -> k is not in output_symbol_to_dim
  -> reduction_syms is non-empty
  -> continue

This is not solved by the current pass. It needs reduction-range tiling and partial accumulation.

9. Coordinate Jointly Controlled by Two Output Symbols

Example pattern:

device coordinate = floor(h / 2048) + floor(2 * Mod(l, 2048))

Some physical layouts interleave two logical dims into one physical stride – for example a device dimension whose coordinate is a sum of independent terms in h and l. Earlier this was skipped outright because coordinate discovery only accepted a coordinate driven by exactly one output symbol; that silently dropped the coordinate from span analysis entirely, so its span was never checked against MAX_SPAN_BYTES even after tiling other dims.

_coordinate_span_elems computes a conservative span bound by summing each free symbol’s independent contribution, so a coordinate mixing two or more output symbols can be evaluated like one with a single symbol. Plain monotonic terms are checked at endpoints. Mod-wrapped terms are evaluated at a wraparound critical point in the full term expression, preserving scale factors such as 2 * Mod(l, 2048).

A per-symbol term can contain more than one Mod atom on that same symbol with different moduli (e.g. two overlapping/interleaved stick strides both keyed on the same loop variable). The term is evaluated at every such atom’s own critical point plus both domain endpoints, and the true max/min is taken across all candidates – evaluating only at a single critical point derived from the smallest modulus would risk assuming that modulus’s wraparound also maximizes every other Mod term summed alongside it, which is not generally true.

The critical-point trick itself is only exact when a Mod’s argument is the bare symbol. A coefficient on the argument (e.g. Mod(3 * h, 64)) wraps at different symbol values than modulus - 1, and evaluating only at the naive critical point can silently underestimate the span (confirmed: Mod(3 * h, 64) over h in [0, 100) has a true max of 63 at h=21, not the 61 found by evaluating only at h=63). _coordinate_span_elems detects this case and returns None (the same “cannot determine, skip this candidate” signal every caller already handles) rather than trust an unproven bound.

Discovery accepts a coordinate as long as every free symbol is output-controlled (still rejecting any coordinate that also involves a reduction-only symbol) and registers one candidate per contributing host dim, so the combo search can consider splitting them together.

Code flow:

_output_span_candidates_from_op / _input_span_infos_controlled_by_output_dims
  -> output_syms = symbols in coord that are output-controlled
  -> skip only if some free symbol is reduction-only
  -> split_by_symbol = {sym: split_by_host_dim[dim(sym)] for sym in output_syms}
  -> coord_span_elems = _coordinate_span_elems(coord, dep, split_by_symbol)
  -> one SpanOverflowCandidate/InputSpanInfo per symbol in output_syms

Entry Point

The planner entry point is:

plan = plan_span_overflow_tile(op, max_cores=config.sencores)

It returns:

SpanOverflowTilePlan(
    levels=(
        SpanOverflowTileLevel(selected_host_dim=..., split_count=...),
        ...,
    ),
    chunking_infos=(...),
    reason="...",
)

or None if no automatic span-overflow tiling is needed or supported for that op.

span_overflow_groups(graph) consumes the plan, attaches synthetic DimHints, and returns groups shaped like user hint groups:

[([op], [(hint_id, split_count, is_reduction_level), ...])]

is_reduction_level is currently False for automatic span-overflow hints; they tile output ranges, not reduction ranges.

Scope

The production planner handles:

• ComputedBuffer ops with Pointwise data;

• ComputedBuffer ops with non-scalar Reduction data;

• output layout span overflow;

• reduction/BMM input span overflow when the span is controlled by output symbols;

• one or more output host dimensions per op, bounded by _MAX_TILE_DIMS;

• static FixedTiledLayout metadata only.

The planner skips:

• non-ComputedBuffer ops;

• non-FixedTiledLayout ops, including mutation/copy-back intermediate layouts;

• symbolic/dynamic layout metadata;

• scalar/full reductions where op.data.ranges is empty;

• Pointwise or Reduction ops with indirect/gather/scatter-style reads;

• input spans controlled only by reduction symbols.

Support Matrix

Case	Supported?	Planner behavior
Pointwise output span overflow	Yes	Emit one or more output-range tile levels
Pointwise indirect/gather read	No	Return None; indirect path owns it
Reduction output span overflow	Yes	Emit output-range tile levels if post-tile layout validates
Reduction input span controlled by output dim	Yes	Emit tile for the matching output dim
Coordinate jointly controlled by 2+ output symbols	Yes	Emit a candidate for each contributing dim
Reduction input span controlled by reduction dim	No	Skip candidate; reduction-range tiling is future work
BMM input span controlled by b, m, or n	Yes	Emit tile for matching output dim
BMM input span controlled by k	No	Skip candidate; k is reduction-only
BMM with ambiguous reduction symbols	No	Return no BMM input candidates
Scalar/full reduction	No	Return None
Symbolic layout metadata	No	Return None
More than _MAX_TILE_DIMS overflowing host dims	No	Raise Unsupported
No legal exact divisor / stick-safe split	No	Raise Unsupported
User already supplied spyre_hint for op	Manual wins	Auto hint is not added for that op

Pointwise Flow

For Pointwise ops, plan_span_overflow_tile does:

check op/layout eligibility
check _has_indirect_reads(op); skip if true
collect output span candidates
search for cheapest valid tile combination

The main output collector is _output_span_candidates_from_op. It prefers physical address-coordinate analysis:

1. read the output MemoryDep;

2. map output iteration symbols to output dimensions;

3. compute physical device coordinates for the output layout;

4. walk non-stick device coordinates;

5. create one SpanOverflowCandidate per output symbol that controls a coordinate whose span exceeds MAX_SPAN_BYTES – a coordinate jointly controlled by several output symbols contributes one candidate per contributing dim (see Coordinate Jointly Controlled by Two Output Symbols).

If output write-dep or symbol-coordinate analysis is unavailable, the planner returns no output candidates instead of falling back to stride-map guessing. This keeps span planning on the same MemoryDep address-math model used by work division while avoiding compile failures for ops that may not need automatic tiling.

Reduction Flow

For Reduction ops, plan_span_overflow_tile does:

check op/layout eligibility
check _has_indirect_reads(op); skip if true
skip scalar reductions
collect output span candidates
collect input span candidates controlled by output dims
search for cheapest valid tile combination

A reduction can overflow in two ways:

1. the output layout itself is too large; or

2. the output is small, but reading an input spans too much physical memory.

For input reads, _input_span_infos_controlled_by_output_dims walks the input physical device coordinates. For each non-stick coordinate it separates free symbols into:

• output symbols: symbols present in _output_symbol_to_dim(op);

• reduction symbols: symbols not present in that output map.

If every free symbol in a coordinate is output-controlled, output-range coarse tiling can reduce that input span, so the planner creates one input-derived candidate per contributing output symbol – one for a coordinate driven by a single output dim, or several for a coordinate that jointly mixes multiple output dims onto one physical stride. If a coordinate involves a reduction symbol, the planner skips that coordinate entirely and continues scanning later device coordinates. That continue behavior is important: a reduction-controlled outer coordinate must not prevent discovery of a more-inner output-controlled overflowing coordinate.

Reduction-only input span overflow remains a known limitation. It requires reduction-range tiling and partial accumulation, which this pass does not implement.

BMM-Specific Symbol Mapping

Batch matmul reductions use BATCH_MATMUL_OP and are treated specially when mapping output symbols:

C[b, m, n] = reduce_k A[b, m, k] * B[b, k, n]

_bmm_output_symbol_to_dim starts from _output_symbol_to_dim, then checks input dependencies to identify exactly one reduction-only symbol (k). If there is not exactly one such symbol, it returns {} and the BMM input-span path produces no candidates.

BMM input spans controlled by b, m, or n can be fixed by output-range tiling. Spans controlled by k are skipped as reduction-range-only.

Span Calculation

For a physical device coordinate at device_dim, span is computed as:

per_core_span = (
    coord_span_elems
    * math.prod(device_size[device_dim + 1:])
    * dtype.itemsize
)

This includes the selected physical coordinate and all inner physical dimensions, including the stick dimension. It does not include outer physical dimensions because those choose separate base regions; each outer coordinate is checked as its own candidate when it varies.

Example:

device_size = [A, B, C, stick]

• checking A: span is A * B * C * stick * itemsize;

• checking B: span is B * C * stick * itemsize;

• checking C: span is C * stick * itemsize.

The current planner uses core_split_estimate=1 for candidate detection and post-tile validation. This models the worst case where Work Division provides no additional help for that coordinate.

Candidate Collection

Each overflow becomes a SpanOverflowCandidate containing a ChunkingInfo:

ChunkingInfo(
    total_bytes=...,
    per_core_span=...,
    core_split_estimate=1,
    selected_device_dim_size=...,
    selected_device_span_stride_elems=...,
    selected_host_dim=...,
    stick_elems=...,
    reason="output span overflow" or "input span overflow for arg0",
)

Candidates can come from:

• output layout span analysis;

• reduction input span analysis;

• BMM input span analysis.

A single physical coordinate jointly controlled by several output symbols produces one candidate per contributing host dim, all sharing the same per_core_span (the real joint span of that one coordinate). Requiring a dim’s own split count from _candidate_required_split_count on such a candidate is a conservative starting estimate – it assumes that dim alone must clear the whole joint span – but it only seeds the bounded divisor search; the post-tile combo is always validated exactly, so an imprecise estimate costs extra combo attempts, not correctness.

_candidate_host_dims merges candidates by host dim and orders dims by decreasing span pressure. This ordering affects the bounded combo search when costs tie.

Split Candidates and Cost Search

For each candidate host dim, _split_candidates_for_host_dim enumerates legal split counts from exact divisors of op.data.ranges[host_dim].

A split candidate is legal only if:

• it divides the selected host range exactly;

• it does not cut through physical sticks in the output layout;

• it does not cut through physical sticks in any fixed-layout input dependency that is addressed by the same output symbol.

The input-layout stick check matters for BMM/transposed operands. A split can look safe on the output layout but misalign an input tensor whose physical stick dimension maps differently.

The search is bounded:

_MAX_TILE_DIMS = 3
_MAX_SPLITS_PER_DIM = 16
_MAX_TILE_COMBOS = 512
_MAX_AUTO_TILE_SPLIT_COUNT = 64

If more than _MAX_TILE_DIMS host dimensions overflow, the planner raises Unsupported. Split candidates per dim are capped before Cartesian product construction, and automatic split counts above _MAX_AUTO_TILE_SPLIT_COUNT are rejected because current automatic LoopSpec lowering can materialize one SDSC spec per coarse tile. A plan such as split_count=769 can therefore generate sdsc_0.json through sdsc_768.json instead of one compact symbolic tile loop. Manual spyre_hint remains explicit/user-controlled and can rely on the more mature user-hint grouping path; the automatic pass is intentionally capped until auto groups can reuse the same compact grouped lowering. Combinations are tried in increasing cost order:

cost = (
    product(split_counts),
    number_of_tiled_dims,
    max(split_counts),
    split_counts,
)

So the planner prefers fewer total tiles, fewer tiled dimensions, and smaller maximum split counts.

Post-Tile Validation

A combo is accepted only after rebuilding the hypothetical per-tile Spyre layout and re-running span analysis.

tiled_layout = _post_tile_layout_for_splits(
    op.layout,
    split_by_host_dim,
    op.get_name(),
)

_post_tile_layout_for_splits changes one or more host sizes, recomputes contiguous host strides, and calls _resize_device_layout to rebuild the real SpyreTensorLayout for that tile.

Then _remaining_span_candidates_after_tile checks:

• output spans using the rebuilt output layout and output MemoryDep.ranges shrunk to the hypothetical per-tile domain;

• reduction/BMM input spans using the candidate split map.

Only if no output or relevant input spans remain above MAX_SPAN_BYTES does the planner return a SpanOverflowTilePlan.

This is the key safety property: the accepted tile combination is validated against the same kind of physical layout that coarse tiling will create.

Multi-Level Plans

The planner can emit multiple output-range tile levels for one op:

SpanOverflowTilePlan(
    levels=(
        SpanOverflowTileLevel(selected_host_dim=0, split_count=2),
        SpanOverflowTileLevel(selected_host_dim=1, split_count=4),
    ),
    ...,
)

Levels are emitted outer-to-inner by host dimension. The adapter creates one synthetic DimHint per level. coarse_tile then stamps a multi-level CoarseTileInfo and scheduler/codegen lower it as nested counted loops.

Adapter and Coarse Tiling

coarse_tile.span_overflow_groups(graph):

1. skips auto groups if config.ignore_wsr_hints or config.ignore_span_overflow_hints is enabled;

2. skips ops that already have user dim hints;

3. calls plan_span_overflow_tile;

4. maps each selected_host_dim to a concrete output loop symbol via op_out_coords(op);

5. fuses a contiguous run of Pointwise ops into one shared group/loop when either (a) each op’s own independent plan produces the exact same (host_dim, split_count, is_reduction) signature as the run so far, or (b) an op’s own plan disagrees but the op directly reads a buffer written by the open run and the run’s split is also legal and sufficient for that op on its own (can_conform_pointwise_tile) — the op then adopts the run’s split instead of its own. Reduction/BMM ops are never grouped or used as a conform target in this pass and always get an independent singleton group;

6. rejects any op that reads a buffer from an already-closed auto-tiled group, from a producer already tiled by a user spyre_hint (checked via the same dim_hints attribute assign_dim_hints leaves behind, since this pass never clears it), or from the open run without being fusable into it (mismatched signature and conform fails, or the reading op is a Reduction/BMM), since two independent loop nests over the same span-overflow-sized data can desynchronize, and materializing a tiled Pointwise producer’s full buffer for such an “outside consumer” can reintroduce the exact span violation tiling was meant to prevent;

7. creates synthetic DimHints with ids starting at _SPAN_OVERFLOW_HINT_ID = 10000, shared across every op in a fused group;

8. returns coarse-tile groups in the same format as user hints.

From coarse_tile onward, automatic and manual hints share the same path:

• rewrite op.data.ranges and op.layout.size per tile;

• stamp CoarseTileInfo;

• wrap scheduled nodes in CountedLoopSchedulerNode;

• emit LoopSpec in codegen.

Configuration

Automatic span-overflow hints are opt-in. By default:

config.ignore_span_overflow_hints == True

because SPYRE_INDUCTOR_IGNORE_SPAN_OVERFLOW_HINTS defaults to 1. Enable the pass with:

SPYRE_INDUCTOR_IGNORE_SPAN_OVERFLOW_HINTS=0

The broader working-set-reduction hint switch still suppresses this path:

config.ignore_wsr_hints == True

User-authored spyre_hint groups take precedence per op. Automatic hints are not added to ops that already carry user dim hints.

When span-overflow hints are explicitly enabled, copy-back elision preserves Pointwise FixedTiledLayout producers instead of rewriting them to mutation layouts. This mirrors the old large-tensor chunking behavior: enabling the automatic span-overflow feature opts into keeping pointwise producer layouts visible to the span planner. With the default disabled setting, normal copy-back elision behavior is unchanged.

Failure Policy

The planner returns None for unsupported-but-safe-to-ignore cases such as ineligible op types, missing output address metadata, failed symbol/coordinate analysis before any overflow candidate is proven, symbolic layouts, indirect reads, scalar reductions, or no span overflow.

It raises Unsupported when it detects overflow but cannot represent a safe automatic output-range tile plan. Common reasons:

• too many overflowing host dims for the bounded search;

• selected host dim is not in op.data.ranges;

• selected range is size 1 or non-integral;

• no legal exact divisor exists;

• stick alignment rejects all candidates;

• _resize_device_layout cannot reconstruct the post-tile layout;

• every tried combination still leaves output/input spans above the limit;

• an automatically tiled op reads a producer that was already automatically tiled, which would require producer-consumer loop fusion to be correct.

These failures are deliberate. They avoid silently emitting a plan that still violates the hardware span limit or silently creates unsynchronized tile loops.

Known Limitations

• Reduction-range tiling is not implemented. Input spans controlled only by reduction symbols are skipped. If such a skipped span still exceeds the hardware limit, downstream Work Division currently logs a critical overflow diagnostic but does not raise before backend compilation.

• Scalar/full reductions are skipped.

• Indirect/gather/scatter-style Pointwise and Reduction ops are skipped because they require the indirect-access SDSC path.

• The combo search is bounded by _MAX_TILE_DIMS, _MAX_SPLITS_PER_DIM, and _MAX_TILE_COMBOS; very high-rank overflow cases can raise Unsupported.

• Automatic split counts are additionally capped by _MAX_AUTO_TILE_SPLIT_COUNT. This is a lowering/codegen safety limit, not a mathematical span limit: current automatic coarse tiling can unroll a large tile count into many generated SDSC specs, which is expensive and can appear to hang compilation. For example, an automatic split of 769 may emit 769 per-tile specs. Manual spyre_hint is still user-controlled and may be able to use existing grouped hint lowering more effectively; automatic span-overflow should grow producer-consumer/grouped loop lowering before this cap is relaxed.

• Reduction auto-tiling rejects splits that shrink the selected output dim’s per-tile extent to 1, because the reduction lowering/DDC template path can drop unit-size iteration dims before fixed-arity template matching. This restriction is intentionally scoped to Reduction ops; Pointwise full-size exact divisors remain legal.

• Symbolic layout metadata is skipped because exact divisibility and post-tile Spyre layout validation require concrete sizes.

• A contiguous run of Pointwise ops fuses into one shared loop, either because each op’s own independent plan already agrees, or because a disagreeing op directly reads the run and can legally conform to the run’s split (can_conform_pointwise_tile in span_overflow_hint_analysis.py). This is still scoped to Pointwise-to-Pointwise: Reduction/BMM ops are never grouped and never conform, and fusion never crosses an already-closed group (closed groups are, by construction, no longer contiguous with what follows). If a Reduction/BMM op reads a producer that was already auto-tiled, or already manually tiled by a user spyre_hint — or a Pointwise op reads one that it cannot conform to — the adapter still raises Unsupported instead of emitting two independent loop groups. This is required for correctness: a restickify/layout-conversion producer and its BMM/LM-head consumer must share one synchronized tile loop, and materializing the producer’s full buffer for such an unfused consumer can reintroduce the exact span violation tiling was meant to prevent. A producer and consumer that are both inside the same manual spyre_hint group are unaffected, since users can explicitly group them into one shared coarse-tile group; the conflict check only fires when an automatically-tiled op reads a manually-hinted producer that it was not itself grouped with. Automatic Reduction/BMM producer-consumer loop fusion, and fusion across an already-closed group, remain future work. A typical failure still looks like Cannot auto-tile buf0: it reads already auto-tiled producer(s) ['buf1'] — now for a narrower set of cases (e.g. very large F.linear/LM-head shapes, where the consumer is a BMM reduction).

• The planner does not yet model expected Work Division splits when choosing coarse-tile counts. Candidate detection uses core_split_estimate=1, so coarse tiling must make spans safe by itself. This is conservative and avoids depending on a later pass, but it can overestimate the coarse split needed when Work Division would already split the same high-pressure coordinate. Future cost-model work should validate the combined effect of planned coarse tiles plus committed/estimated Work Division splits, so auto coarse tiling only covers residual span pressure and avoids excessive LoopSpec counts.

• The combo search only considers host dims found by the initial untiled candidate scan. Post-tile validation uses the per-tile output ranges and rebuilt physical layout, so it validates the same domain the tiled kernel will execute. Widening the search to absorb new dims discovered only during re-validation remains future work if such a case appears.

• Mutation/copy-back intermediate layouts are currently outside planner scope. When span-overflow hints are enabled, pointwise copy-back elision preserves FixedTiledLayout producers so the planner can still see normal pointwise producers. Direct planning through MutationLayoutSHOULDREMOVE remains a future eligibility question.

• A coordinate containing a Mod atom whose argument is not the bare split symbol (e.g. Mod(3 * h, 64), as opposed to Mod(h, 64)) is skipped rather than bounded: _coordinate_span_elems returns None for it, since the critical-point trick used for bare-symbol Mod atoms is not exact once a coefficient is applied inside the Mod. No concrete op/lowering path has been traced that produces this shape for a FixedTiledLayout device coordinate today; this is a documented fail-safe boundary rather than a known-reachable gap.

Validation

Main unit/codegen coverage lives in:

tests/inductor/test_span_overflow_hint_analysis.py

Run:

python3 -m pytest -q torch-spyre/tests/inductor/test_span_overflow_hint_analysis.py

Current coverage includes:

• no-op behavior under span limit;

• Pointwise output span tiling;

• Reduction output span tiling;

• Reduction/BMM input span tiling;

• scalar reduction skip;

• indirect-read guards for Pointwise and Reduction;

• reduction-controlled span known limitation;

• BMM reduction-symbol validation;

• input-layout stick alignment, including a symbol that jointly controls an input dimension together with another symbol rather than controlling it alone;

• multi-level plan generation;

• coordinates jointly controlled by two output symbols producing one candidate per contributing dim, on both the output and reduction/BMM input paths;

• Mod-wrapped coordinate span bounds that preserve coefficients around the wraparound point, correctly bound a term with multiple Mod atoms on the same symbol at different moduli, and fail safe (return None) for a Mod whose argument is not the bare symbol;

• post-tile validation using per-tile output ranges;

• adapter and coarse_tile stamping;

• fail-safe rejection for the LM-head pattern where an auto-tiled restickify producer feeds an auto-tiled BMM consumer;

• codegen LoopSpec tests for Pointwise, Reduction, and LM-head restickify shapes.

Key Files

File	Role
torch_spyre/_inductor/span_overflow_hint_analysis.py	Candidate collection, combo search, post-tile validation, tile-plan dataclasses
torch_spyre/_inductor/coarse_tile.py	Adapter from tile plans to synthetic DimHints; coarse-tile IR stamping
torch_spyre/_inductor/passes.py	Combines user hint groups and automatic span-overflow groups
torch_spyre/_inductor/propagate_layouts.py	Preserves pointwise producer layouts from copy-back elision when automatic span-overflow is explicitly enabled
torch_spyre/_inductor/ir.py	Spyre layout resize/reconstruction helpers
torch_spyre/_inductor/work_division.py	Span limit constants and downstream per-core span diagnostics
tests/inductor/test_span_overflow_hint_analysis.py	Main unit/codegen coverage