vLLM extends an INT4 KV-cache attention shape path
By OnTheRice · Sources and limitations · Report a correction
The INT4 per-token-head KV-cache path uses a block-diagonal Hadamard transform based on the largest power-of-two divisor of non-power-of-two head size. Backend-selection tests cover head sizes 64, 80, 96, 128, 192 and 256 and expect TRITON_ATTN in the tested CUDA configuration. OTR sees: The implementation targets attention geometry that does not divide evenly into a power-of-two head dimension, widening the set of tested INT4 KV-cache shapes accepted by the path. The change is a bounded code-path update; broader rollout, frequency or performance effects are not established.
Source confidenceNot assessed
What’s changing?
A block-diagonal transform using the largest power-of-two divisor provides a defined route for non-power-of-two head dimensions in the INT4 cache path.
Key points
- The INT4 per-token-head KV-cache path uses a block-diagonal Hadamard transform based on the largest power-of-two divisor of non-power-of-two head size.
- Backend-selection tests cover head sizes 64, 80, 96, 128, 192 and 256 and expect TRITON_ATTN in the tested CUDA configuration.
- OTR sees: The implementation targets attention geometry that does not divide evenly into a power-of-two head dimension, widening the set of tested INT4 KV-cache shapes accepted by the path.
- The change is a bounded code-path update; broader rollout, frequency or performance effects are not established.
What could change this view?
- The test’s CUDA setup and listed shapes are bounded, so support should not be generalized to every model or runtime.
Bottom line
Backend eligibility changes for the tested head-size class; real model coverage and performance are still unproven.
This is based on the information recorded with this signal.
How likely is this call? · Three frameworks
This signal has no investible bullish or bearish call. A market-direction percentage does not apply.
Evidence behind this view
Checks whether separate pieces of evidence support the same view.
Recorded connection
A block-diagonal transform using the largest power-of-two divisor provides a defined route for non-power-of-two head dimensions in the INT4 cache path.
We have not established a separate count of independent evidence clusters for this card. Several reports may rely on the same event or data provider.
What past price moves show
Compares today’s price moves with recorded past patterns.
This card has no recorded investible call. A price-pattern comparison is not applicable unless a specific asset is added with evidence.
Evidence against this view
Shows evidence and risks that could challenge this view.
Recorded opposing evidence and risks
The test’s CUDA setup and listed shapes are bounded, so support should not be generalized to every model or runtime.
Why the direction changed
Explains recorded changes from expecting a rise to expecting a fall, or the reverse.
No documented bullish-to-bearish or bearish-to-bullish change is available for this card. We keep the original call and its time unchanged.
What people say vs the facts
Compares recorded public discussion with checked market facts.
No separate Reddit or public-discussion comparison with verified market data is recorded for this card. We cannot treat social attention as proof of a market move.
How past forecasts performed
Shows recorded forecast results and what they can teach us.
This card has no recorded investible forecast to score.
No tested mapping from evidence combinations to future success is recorded for this card. Confidence in sources and past outcomes do not establish a calibrated probability.
Sources, original wording and price checks
Published
First published through Sites 05 Oct 2026, 14:42:21 SGT
No explicit investible BULLISH or BEARISH call on this card.
What changed
Recorded public historyThe INT4 per-token-head KV-cache path uses a block-diagonal Hadamard transform based on the largest power-of-two divisor of…
This is the first recorded public view. Further public updates will appear here.
- vLLM extends an INT4 KV-cache attention shape path
The INT4 per-token-head KV-cache path uses a block-diagonal Hadamard transform based on the largest power-of-two divisor of non-power-of-two head size. Backend-selection tests cover head sizes 64, 80, 96, 128, 192 and 256 and expect TRITON_ATTN in the tested CUDA configuration. OTR sees: The implementation targets attention geometry that does not divide evenly into a power-of-two head dimension, widening the set of tested INT4 KV-cache shapes accepted by the path. The change is a bounded code-path update; broader rollout, frequency or performance effects are not established.
First published by OTR: 5 Oct, 14:42 SGTPublished by OTR: 5 Oct 2026, 14:42 SGT
Why the view changed
This is the first recorded view. Future source-backed updates will appear here.
Evidence, memory & review 1 linked source
Directional takeaway: ABSTAIN
OTR reasoning: A block-diagonal transform using the largest power-of-two divisor provides a defined route for non-power-of-two head dimensions in the INT4 cache path.
Counterpoint: The test’s CUDA setup and listed shapes are bounded, so support should not be generalized to every model or runtime.
Evidence basis: Source-checked editorial summary; this is a paraphrase and synthesis, not a source quotation.
Source trail
- vLLM extends an INT4 KV-cache attention shape path ↗GitHub / project repository2026-10-05T04:52:39Z · published (UTC)
Review status: PASS WITH COMMIT DIFF SCOPE AND NO UNMEASURED EFFECT CLAIM
No score or probability is presented.