TAG
Zone setting changes used to rebuild entire DNS zones, even ones with tens of millions of records. Here is how we built a planner that picks the smallest safe build, cutting full builds by 40%, peak memory by 85%, and halving build delay.
Cloudflare Security Insights system now processes over 120 scans per second, providing frequent insights for all customers. By optimizing Kafka consumers, Postgres queries, and our API, we scaled our throughput 10x without adding hardware.
DNS records have moved to a new database, bringing improved performance and reliability to all customers.
In response to customer demand, we now support the ability to DELETE, PATCH, PUT and POST multiple DNS records in a single API call, enabling more efficient and reliable zone management.
Announcing the new Upstash database integrations for Workers. Now it is easier to use Upstash Redis, Kafka and QStash inside your Worker
At Cloudflare, we take steps to ensure we are resilient against failure at all levels of our infrastructure. This includes Kafka, which we use for critical workflows such as sending time-sensitive emails and alerts.
We learnt a lot about Kafka on the way to 1 trillion messages, and built some interesting internal tools to ease adoption that will be explored in this blog post
How we redesigned our DNS pipeline to significantly improve DNS propagation speed across all zones.
A refresh of the hardware that Cloudflare uses to run analytics provided big efficiency improvements.
How an innocent OS upgrade triggered a cascade of issues and forced us into tracing Linux networking internals.
One of our large scale data infrastructure challenges here at Cloudflare is around providing HTTP traffic analytics to our customers. HTTP Analytics is available to all our customers via two options:
How Cloudflare was able to save hundreds of gigabits of network bandwidth and terabytes of storage from Kafka.
When I joined CloudFlare about 18 months ago, we had just started to build out our new Data Platform. At that point, the log processing and analytics pipeline built in the early days of the company had reached its limits.
