AI signal, minus the noise.

A user measured input token costs for an AI agent browsing similar pages over 20 turns. Turn 1 consumed roughly 300 tokens, while turn 20 consumed 7,000 tokens—a 20× increase—as the agent re-reads all previous context. The observation highlights a hidden “context tax” that drives up inference costs in multi-turn agent workflows.

Release b9626 of llama.cpp introduces support for the Cohere2 Mixture of Experts (MoE) architecture under the new arch name "cohere2moe". It fixes sliding window attention pattern handling, resolves MTP failures by switching to iSWA, and adjusts shared expert combination to (routed+shared)*0.5. Redundant gating function checks, lmhead tensor checks, and tokenizer type definitions were removed; the tokenizer is kept as tiny_aya. Platform builds are provided for macOS (Apple Silicon/Intel), Linux (x64/arm64 with Vulkan, ROCm, OpenVINO, SYCL), Android, and Windows (CPU/CUDA/Vulkan/SYCL/HIP), along with UI support.

llama.cpp version b9625 has been released. The release contains a fix for jinja template rendering that failed when slicing with a negative step and start/stop values (PR #24580). Pre-built binaries are published for macOS Apple Silicon (arm64), Ubuntu Linux (x64 and arm64) with various backends including CPU, Vulkan, ROCm, OpenVINO, and SYCL, Windows (x64, arm64) with CUDA 12/13, Vulkan, SYCL, HIP, and Android arm64. Builds for macOS Intel, iOS, and openEuler are listed as disabled.

Release b9624 of llama.cpp introduces build-time gzip compression for the web UI and resolves a nocache issue to preserve original file names and paths. Pre-built binaries are provided for macOS (Apple Silicon with KleidiAI disabled, Intel x64), Linux (multiple CPU and GPU backends), Windows (CPU, CUDA 12/13, Vulkan, SYCL, HIP), Android arm64, and openEuler platforms. This release also extends CI coverage across many configurations.

The open-source LLM inference project llama.cpp has released version b9623. This release includes a fix for Jinja template functions `split` and `replace`, which previously mishandled cases where the first argument was an empty string. Additionally, a bug in reserve size calculation was corrected. The release notes also list the current build status across various platforms, including macOS, Linux, Android, and Windows configurations.

A V2EX user reported that a friend purchased a GLM annual subscription as a backup while primarily using OpenAI's Codex and ChatGPT. After recent policy-driven access restrictions (possible reference to “Fable” or similar incidents), that backup proved strategically valuable. The user warns against sole dependence on providers like OpenAI or Anthropic, whose policies can cut off access without notice, and plans to similarly secure a GLM annual plan. The post highlights growing community concerns over API dependency and the importance of having fallback options.