## Making Workers AI faster and more efficient: Performance optimization with KV cache compression and speculative decoding

2024-09-26

With a new generation of data center accelerator hardware and using optimization techniques such as KV cache compression and speculative decoding, we’ve made large language model (LLM) inference lightning-fast on the Cloudflare Workers AI platform....