AI reuse needs rules for work and shared memory

Original research record

AI reuse needs rules for work and shared memory

We connect new PyTorch, XLA and vLLM changes around one question: what may a system safely reuse? PyTorch reuses a compiled plan while keeping required logging. XLA accepts an explicit map showing when an output shares an input buffer. vLLM keeps dummy requests away from a cache block they do not own. We infer that reuse tests need to check both work that must still happen and storage that must stay untouched.

We see two different costs when software loses the meaning of a reusable object. It can repeat work unnecessarily, or allow a write that should never happen. In PyTorch, an ordered-effect wrapper previously forced the graph cache to be skipped. The new validator checks the operation inside the wrapper. A permitted cache hit reuses the compiled plan, while the test still requires the logging calls to run in the same order. It does not reuse a past logging result. [PyTorch source](https://github.com/pytorch/pytorch/commit/7b95b29df235ff10f08229bb4966fe9798b264a1)

XLA addresses storage rather than compiled plans. Some asynchronous operations update an existing input buffer. A new builder option preserves the explicit input/output sharing map. Its tests show the analysis recognising that link and avoiding a second logical output buffer. This is source-level evidence about memory planning, not a measured reduction in a full model’s GPU memory. [XLA source](https://github.com/openxla/xla/commit/d9d1309fc7837c426c52b8cdb3e3463caef9e81d)

vLLM shows the opposite boundary. In two attention-cache paths, block zero means a null block owned by no real request. Dummy and padding requests must map to a no-write slot, rather than treat zero as usable storage. The new code and tests enforce that rule. We connect these separate changes into a practical acceptance test: show what is reused, what still runs and what cannot be written. They are not one shared defect. [vLLM source](https://github.com/vllm-project/vllm/commit/240785b82c299c6c08bc5239e997cf62d67cf7bb)

Key points We would separate a compiled-plan cache hit from reuse of a previous answer. PyTorch’s test checks identical outputs and ordered logging on both the first run and the later cache hit. We read XLA’s builder and analysis tests. The new mapping tells the compiler that the asynchronous output may share a specific input. The test checks logical buffer definitions, not a hardware memory meter. We checked vLLM’s dummy-batch and mixed real/padding cases. The null-block rows map to -1, the no-write marker. The change does not say that every zero-valued index in vLLM is invalid. PyTorch’s author reports 100–165 seconds per rank of avoidable compilation in one internal Ads training workload. We did not access or reproduce that private result and do not apply it to other workloads.

What could change this view? We would narrow this view to the affected code paths. PyTorch still refuses cache reuse for a wrapped operation that is not cacheable, and its new test checks this. XLA adds a way for callers to supply shared-buffer information; it does not make every asynchronous operation reuse memory automatically. vLLM’s change covers the named GLM-5.3 and Qwen4 cache paths. We have no whole-model memory benchmark, production failure rate or verified containing release. Three selected source changes do not show an industry-wide trend.

Bottom line We would accept a reuse gain only after checking the work it preserves and the storage it is allowed to touch. Test required logging on a plan-cache hit, the exact input/output buffer link and dummy requests that must not write. Measure actual time and memory separately before budgeting a saving.

How likely is this call? · Three frameworks This signal has no investible bullish or bearish call. A market-direction percentage does not apply. Bayesian · what changed the estimate? We add exact evidence about allowed plan reuse, declared buffer sharing and forbidden null-block writes. We have no fitted prior or likelihood for their practical benefit, so we make no numerical probability update. de Finetti · are the comparisons fair? Compiled plans, logical buffers and request cache slots are different objects. These selected changes are not interchangeable trials or three replications of one result, so we do not pool them into a success rate. Kolmogorov · do the numbers agree? We give no bullish, bearish or neutral percentages. A cache hit, a logical buffer count and a no-write marker are not probabilities. No market outcome partition applies. Horizon, exact deadline, neutral band and price basis: Not applicable. Read the shared method: https://new.ontherice.org/how-otr-works/#probability-framework.

Evidence behind this view We connect OTR Database Data Technology attempts 96 and 97, then reopen all three exact source changes. vLLM was committed at 20:44:04 UTC on 8 October, PyTorch at 21:36:16 and XLA at 21:58:32. We observed the primary receipts at 22:49:47–22:49:51 UTC. These are three project-origin clusters, with complete returned changed-file patches read.

What past price moves show Not applicable to market prices. Earlier OTR cards examined cold/warm correctness and saved state after a restart. This view adds a different mechanism: explicit rules that permit compiled-plan or input-buffer reuse while blocking writes to unowned storage. It does not replace or resolve those earlier records.

Evidence against this view We would narrow this view to the affected code paths. PyTorch still refuses cache reuse for a wrapped operation that is not cacheable, and its new test checks this. XLA adds a way for callers to supply shared-buffer information; it does not make every asynchronous operation reuse memory automatically. vLLM’s change covers the named GLM-5.3 and Qwen4 cache paths. We have no whole-model memory benchmark, production failure rate or verified containing release. Three selected source changes do not show an industry-wide trend.

Why the direction changed No market direction changes and no locked forecast is revised. We add a new connection between three source changes with their real commit clocks. The vLLM commit predates the prior first website publication; PyTorch and XLA were committed later. New research is not the same as three events happening at publication time.

What people say vs the facts We have not sampled public developer discussion or measured user adoption for this card. The source tests support specific rules. They do not show that most AI systems waste memory or suffer cache corruption.

How past forecasts performed No matched forecast set or deployment denominator measures how often these rules improve a workload. We provide no success probability, market return, cost-saving estimate or historical accuracy claim. Reading test definitions is not running them.