Everything published in July 2026.

Articles from July 2026

• Inference Systems Jul 24, 2026 ·12 min read

  Transformer Inference, From HTTP Request to the Next Token

  A systems-level tour of prefill, decode, batching, memory movement, and the scheduling decisions that determine real-world LLM latency.

• Inference Systems Jul 18, 2026 ·12 min read

  KV Cache Engineering for LLM Inference

  How to size, allocate, reuse, compress, and evict the state that makes autoregressive decoding practical.

• Inference Systems Jul 12, 2026 ·13 min read

  Semantic Caching Without Serving the Wrong Answer Faster

  A production design for similarity keys, freshness, authorization, invalidation, and the economics that determine when semantic reuse is safe.

• Inference Systems Jul 10, 2026 ·12 min read

  Speculative Decoding Without Hand-Waving

  Draft models, acceptance math, tree proposals, and the operational details that decide whether speculation makes serving faster.

• Evaluation and Security Jul 9, 2026 ·13 min read

  Synthetic Evaluation Data That Finds Real Failures

  How to generate, filter, diversify, and maintain synthetic cases without turning an evaluation suite into a mirror of its generator.

• Inference Systems Jul 6, 2026 ·13 min read

  The LLM Gateway Is a Policy Engine, Not a Proxy

  Designing the routing, budgets, identity, resilience, and evidence layer between products and a changing model portfolio.

• Evaluation and Security Jul 3, 2026 ·13 min read

  Privacy-Preserving AI Is a Dataflow Architecture

  Purpose limitation, minimization, isolation, retention, redaction, and verifiable deletion across retrieval, models, tools, traces, and feedback loops.