Feature
Pingora now powers Cloudflare's cache
Cloudflare's cache now runs on a new proxy built on Pingora, the Rust-based framework that already serves a significant portion of Cloudflare's network traffic. The new proxy is faster, more memory-safe, and designed to evolve our cache architecture. It delivers immediate performance improvements and enables new caching capabilities.
What this brings
- Lower latency: The new proxy reduces per-request overhead through improved connection reuse.
- Reduced cache MISSes: Enhanced cache retention improves origin offload.
- Better RFC compliance: Caching behavior more closely follows HTTP caching standards.
- Foundation for future features: The new architecture enables upcoming improvements to cache functionality and efficiency.
New features
- Asynchronous stale-while-revalidate: Every request returns stale content immediately while revalidation happens in the background, instead of the first request after expiry blocking on the origin. Refer to the asynchronous stale-while-revalidate changelog for details.
- Unbuffered bypass by default: Responses that bypass cache are streamed directly to the client without buffering, reducing time-to-first-byte for uncacheable content.
Behavioral changes
The new architecture introduces the following behavioral changes to improve RFC compliance and correctness:
- Vary: * results in cache bypass: According to RFC 9110 Section 12.5.5, a Vary header value of * indicates the response varies on factors beyond request headers and must not be served from cache. Cloudflare now bypasses cache for these responses instead of storing them.
- Set-Cookie stripped on MISS and EXPIRED: For cacheable assets, Set-Cookie is now stripped on MISS and EXPIRED responses, not only on HITs.
- Floating-point TTL values: Floating-point time-to-live values (for example, max-age=1.5) are rounded down to the nearest integer instead of being rejected as invalid.
What's next
A deeper look at the new cache proxy is coming soon to the Cloudflare blog. For background on the underlying framework, read:
Read the original announcement