· 3 items
OpenAI ↗
Better prompt caching for GPT-6
- What
- OpenAI released improved prompt caching capabilities for GPT-6.
- Why
- It reduces latency and costs via higher cache hit rates and explicit breakpoints.
- Impact
- Lowers token costs for active voice agent development.
arXiv ↗
Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memor…
- What
- Flash-dLLM introduces IO-aware KV caching and parallel decoding for diffusion LLMs.
- Why
- It addresses the lack of effective KV caching in non-autoregressive text generation.
- Impact
- Speeds up non-autoregressive text generation.
arXiv ↗
CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Ag…
- What
- CliffCompaction offers cost-efficient autocompaction for long-horizon coding agents.
- Why
- It reduces context token costs by up to 50 percent while maintaining performance.
- Impact
- Improves long-horizon agent state management.