ALL INSIGHTS

LLM Enhancements

BlockLock: Making Batch Geometry a Control for Reproducible LLM Serving

BlockLock reports higher throughput with reproducible outputs by fixing batch geometry for selected requests, not the entire workload.

November 6, 20269 min read
BlockLock: Making Batch Geometry a Control for Reproducible LLM Serving

LLM serving systems face a tension between flexible execution and repeatable behavior. Dynamic batching lets an engine combine requests as they arrive and advance through generation. But changes in batch composition can also change execution geometry, affecting numerical behavior as well as latency and throughput. For teams that need stable evaluations or repeatable production behavior, that makes batching more than a utilization decision.

BlockLock addresses this problem by fixing batch geometry for selected requests while leaving other requests less constrained. Its paper, “BlockLock: Reproducible LLM Inference with Fixed Batch Geometry,” was submitted to arXiv on November 4, 2026, and updated on November 6, 2026. The central proposal is selective protection: reproducibility-sensitive requests receive fixed geometry without imposing that requirement on the entire workload. [6]

The reported performance is substantial. At equal delivered work, BlockLock-Rows reduces latency by 53.5 to 55.0% and improves throughput by 115.2 to 122.2% relative to vLLM Batch-Invariant. Both evaluated BlockLock modes preserved target sequences and Top20-logit bits in the reported tests. Those results warrant attention, but they also require careful interpretation: the available evidence does not establish universal gains across hardware, models, or production workloads. [6]

1. Batch geometry belongs in the reproducibility discussion

Batch geometry describes how requests and tokens are arranged for model execution. During autoregressive decoding, a serving engine may combine sequences with different lengths and generation states. Requests enter, complete, or are regrouped, changing the tensor shapes and execution paths used across iterations.

Dynamic batching uses this flexibility to pursue high GPU utilization. The complication is that an individual request does not necessarily encounter the same execution arrangement every time it runs. BlockLock targets the resulting relationship between batch geometry and reproducible numerical behavior. Its contribution is not simply a faster way to process an unchanged batch. It is a serving policy that controls which requests can be exposed to geometry changes. [6]

That distinction matters when evaluating repeatability. A team may want to know whether a changed output reflects a model change, a serving-system change, or differences in execution conditions. Batch geometry is one of the conditions that BlockLock explicitly seeks to control. The paper’s output checks therefore matter alongside its performance measurements: both operating modes preserved the tested target sequences and Top20-logit bits. [6]

These checks should not be expanded into a broader guarantee than the evidence supports. Preserving the reported sequences and leading-logit bits does not establish that every intermediate value, every logit, or every possible request remains identical across all deployment conditions. It demonstrates preservation of the properties tested in the evaluation.

For engineers, the useful shift is conceptual. Reproducibility can be treated as a property of serving behavior, with explicit scheduling constraints and measurable output checks, rather than assumed from the model and request alone.

2. Selective protection is the central design choice

A fixed batch layout applied globally could restrict requests that do not need stronger repeatability. BlockLock instead constrains designated requests. The rest of the workload can retain more flexible serving behavior. This selective enforcement is the mechanism that distinguishes the proposal from a blanket requirement for fixed execution geometry. [6]

The design creates a useful policy distinction between protected and unprotected traffic. Protected requests are subject to geometry constraints intended to preserve reproducibility. Unprotected requests are not subject to those same protection constraints. The available brief does not explain the complete scheduling mechanics, so it would be premature to infer how protected requests are placed, admitted, or coordinated internally.

The paper reports two operating modes, with the headline latency and throughput results specifically attributed to BlockLock-Rows. Both modes preserved target sequences and Top20-logit bits in the reported evaluation. The available evidence does not provide enough detail to characterize the other mode or compare their operating trade-offs beyond those shared output checks. [6]

Another practical feature is that BlockLock does not require custom kernel implementations. Its approach acts through execution-geometry constraints rather than requiring a new kernel implementation to obtain the reported behavior. [6] That is relevant to infrastructure evaluation, but it should not be confused with a claim of integration-free deployment. The available source does not establish implementation effort, operational complexity, or compatibility with every serving configuration.

The broader implication is that reproducibility need not be an undifferentiated requirement for all traffic. Teams could evaluate whether regression tests, audit-sensitive requests, or other repeatability-sensitive workloads merit protection while allowing ordinary traffic to retain greater batching flexibility. Those are potential applications of the design, not established deployment results.

3. Read the performance numbers with their boundaries intact

BlockLock’s headline figures describe a specific comparison: BlockLock-Rows against vLLM Batch-Invariant at equal delivered work. Under those evaluated conditions, the paper reports a latency reduction of 53.5 to 55.0% and a throughput improvement of 115.2 to 122.2%. The throughput result corresponds to more than doubling the cited baseline’s throughput. [6]

Three boundaries are essential to interpreting those numbers.

First, the baseline is specific. The result is not a comparison against every vLLM configuration. It does not show that BlockLock-Rows provides the same improvement over an unconstrained dynamic-batching setup, another serving engine, or a differently configured reproducibility mechanism. Removing “Batch-Invariant” from the baseline description would materially change the claim.

Second, equal delivered work matters. Comparing systems that complete different amounts of output can obscure the relationship between latency and useful throughput. The paper’s equal-work comparison provides a more meaningful basis for the reported performance difference. However, the available excerpt does not provide the complete accounting methodology, so readers should not infer details about how delivered work was measured. [6]

Third, the configuration remains incompletely described in the available evidence. Exact models, GPU types, request distributions, batch sizes, and workload composition are not specified for every reported result. Those omissions limit how confidently an infrastructure team can translate the percentages into its own environment.

The latency range should also remain described as latency. The brief does not identify it as time to first token, inter-token latency, a particular percentile, or another specific measure. Assigning one of those meanings would add unsupported precision.

The appropriate conclusion is narrower and more useful than a universal speedup claim: BlockLock-Rows reports substantially better performance than the cited reproducibility-oriented baseline while preserving the tested output properties. Whether that advantage transfers to a particular deployment requires workload-specific evaluation. [6]

4. Output preservation is evidence, not an unlimited guarantee

The preservation of target sequences and Top20-logit bits gives BlockLock’s evaluation a second dimension beyond serving speed. The paper checks both the generated sequences and bit-level information associated with the leading logits. Both operating modes preserved those properties in the reported tests. [6]

For a team evaluating reproducible inference, that is more informative than a throughput result alone. Faster execution would not satisfy the intended objective if it changed the outputs that the protection mechanism was supposed to preserve. Conversely, preserving outputs without measuring the performance cost would leave the operational trade-off unresolved. BlockLock reports evidence on both sides.

Still, reproducibility requirements differ. A regression suite may focus on exact generated sequences. Another evaluation may inspect leading-logit information. A production governance process may require additional controls or records that the available BlockLock evidence does not address. Teams should define their acceptance criteria explicitly rather than treating “reproducible” as a complete specification.

The available source also does not establish whether the same results hold for long-context prompts, mixture-of-experts models, multimodal models, or heavily heterogeneous traffic. It provides no basis for claiming hardware-independent or workload-independent preservation and performance. [6]

A careful assessment therefore separates three questions: which output properties were checked, under which serving conditions they were preserved, and what performance accompanied that preservation. BlockLock supplies promising reported answers for its evaluated configurations. It does not eliminate the need to answer those questions again for the environment in which a team intends to use the technique.

5. Geometry control addresses a different bottleneck

BlockLock sits alongside several inference optimization directions, but its results should not be ranked against them as if they were interchangeable benchmarks. The methods target different constraints, use different baselines, and address different execution conditions.

Quantization provides a clear example of why context matters. “Twice as Fast, Until It Isn’t: Quantized Serving of Mid-Size Open LLMs” reports approximately 2.12 times the speed with 4-bit weights for Qwen2.5-14B on an A100 in a small-batch, memory-bandwidth-limited decoding case. For Gemma 3 12B at batch size 256, the reported gain falls to 1.02 times as the GPU becomes compute-bound. The paper identifies FP8 as an option that can still help in that regime. [3] The lesson is not that quantization is ineffective, but that the active bottleneck determines its value.

Scheduling research addresses another part of the system. “Asymptotic Optimality of cμ-Rule for LLM Inference Scheduling under Memory Constraint” models serving as a multiclass queue whose effective service rate reflects computation, memory bandwidth, and memory capacity. Its authors report reduced average latency while maintaining high throughput in NVIDIA H800 experiments. [1] That approach prioritizes requests according to resource-aware scheduling, whereas BlockLock constrains execution geometry for selected requests.

Memory-limited execution offers a third contrast. “Layer-wise CPU-GPU Scheduling for LLM Inference on Memory-Limited Consumer GPUs” overlaps CPU execution of selected layers with asynchronous transfer of subsequent weights to the GPU. It reports 3.5 times acceleration for OPT-30B over HuggingFace Accelerate and 6.3 times over DeepSpeed Zero-Inference in a consumer desktop environment. [5] Its target is memory capacity and transfer latency, not batch reproducibility.

At a different scale, work on workload-static optical circuit switching targets the communication fabric. For a deployment with 576 accelerators serving 5T-parameter mixture-of-experts models with up to 5M tokens of context, it reports improvements of up to 65% in interactivity and 119% in energy efficiency over 4:1 oversubscribed packet-switched fabrics. [7]

Together, these studies reinforce the need to identify the limiting constraint before selecting an optimization. The available evidence does not establish whether their gains combine with BlockLock’s.

6. Turn the proposal into a workload-specific evaluation

For infrastructure buyers and platform engineers, the next step is an evaluation plan rather than an extrapolated savings estimate. BlockLock’s reported gains justify investigating fixed geometry, particularly where reproducibility is already a requirement. They do not yet support a fleet-wide performance or cost projection.

A useful evaluation would begin with the traffic classification implied by the design. Which requests genuinely need protection? What output properties must remain stable? Which requests can remain under more flexible batching? Answering those questions establishes the intended policy before measuring its cost or benefit.

The benchmark should then preserve the distinctions in the paper’s claims:

  • Identify the baseline precisely. Reproduce the relevant comparison with vLLM Batch-Invariant where applicable, and separately compare against the configuration the team currently operates.
  • Hold delivered work comparable. Document the work completed by each configuration rather than relying on throughput figures detached from output volume.
  • Check outputs as well as speed. Include target-sequence and Top20-logit-bit checks if those match the team’s reproducibility requirements.
  • Record workload and hardware conditions. Make model choice, GPU type, sequence lengths, request distributions, and traffic composition explicit.
  • Vary the protected workload. Evaluate protection policies that reflect anticipated use rather than assuming one traffic mix represents production.

These are recommended evaluation practices, not additional results reported by the paper. Their purpose is to resolve the gaps left by the available evidence. A successful test should show not only that a configuration is faster, but that it preserves the required behavior under the conditions the team actually expects to serve.

7. What this means for teams running their own inference infrastructure

BlockLock makes a focused case for treating batch geometry as an explicit infrastructure control. Its selective design recognizes that some requests need stronger repeatability while others benefit from less constrained scheduling. That distinction could be useful for teams combining evaluation, regression testing, and production traffic on their own serving infrastructure. [6]

The reported results are encouraging: substantial latency and throughput improvements against vLLM Batch-Invariant at equal delivered work, preservation of target sequences and Top20-logit bits in the tested modes, and no requirement for custom kernels. Those claims should travel together with their scope. They are findings from evaluated configurations, not guarantees for every deployment. [6]

For teams operating their own stack, the immediate implication is to make reproducibility requirements explicit and test them alongside performance. If fixed geometry is valuable only for a subset of requests, selective enforcement deserves evaluation. The decision should rest on measured behavior in the target environment, with a named baseline and clearly defined output checks.

BlockLock does not settle the performance and reproducibility trade-off. It offers a concrete way to control it at the serving-system level, making execution geometry something teams can evaluate deliberately rather than leave as an incidental consequence of batching.

Sources

  1. [1]Asymptotic Optimality of cμ-Rule for LLM Inference Scheduling under Memory Constraint
  2. [2]Quality-aware cost minimization for multi-tier LLM routing in ...
  3. [3]Twice as Fast, Until It Isn't: Quantized Serving of Mid-Size Open LLMs
  4. [4]Online Quantization: Smooth Path from One Checkpoint to Any Desired Serving Scenario
  5. [5]Layer-wise CPU-GPU Scheduling for LLM Inference on Memory-Limited Consumer GPUs
  6. [6]BlockLock: Reproducible LLM Inference with Fixed Batch Geometry
  7. [7]Workload-Static Optical Circuit Switching For Scalable LLM Inference Decode
  8. [8]Seeing What Matters, Verifying What Pays: Load-Adaptive Speculative Decoding for Vision-Language Models
  9. [9]Ricoh paper on information-theoretic method for steering LLM behavior accepted at AACL 2026 | Global | Ricoh
  10. [10]Speculative Decoding Coordination Protocol (SDCP) for Distributed Inference with Multi-Level Caches

Researched from the sources above and fact-checked against them before publishing.

See what this means for your infrastructure.

Book a discovery call and get a benchmark projection for your hardware and model.

SCHEDULE A CALL