Architecture · Nov 2026 · 4 min read · 01/05
Designing a safe webhook delivery platform
Slow clients, head-of-line blocking, private IPs and DNS rebinding: what makes sending an HTTP request hard at scale.

Last week I was designing a webhook delivery platform, and it turned into one of those problems that looks simple on a whiteboard and gets genuinely hard the moment you think about scale and security.
The setup: consume events off a RabbitMQ topic, worker nodes pick them up, deliver webhooks to client systems. Straightforward, right? Here's where it got interesting.
1. Client returns a 500
Standard fix, retry with exponential backoff. Nothing exotic here, but easy to get wrong if you don't cap max attempts and route eventual failures to a dead letter queue.
2. The slow client problem
This was the real headache. What happens when a client's endpoint takes 30-60 seconds to time out instead of failing fast? Multiply that by 1000 webhooks queued for that one system, and suddenly every healthy client behind it is stuck waiting.
Our fix: tenant-wise queuing, so one slow tenant can't block another. But queuing alone isn't enough. You also need a hard timeout on the delivery call itself (5-10s, not 30-60s) and a circuit breaker that stops hammering a dead endpoint after a few consecutive failures, instead of retrying blindly into a wall.
3. Base URL resolves to a private IP
A client (or an attacker) points a webhook at an internal address. Solution: validate whether the resolved IP is public or private before attempting delivery.
4. The sneaky one: DNS rebinding / SSRF
Here's the attack: you resolve the URL, it comes back public, your validator approves it. Then, right before you connect, the attacker's DNS (set with a very low TTL) flips the record to a private IP. You just got tricked into hitting an internal service.
The fix: resolve once, validate that IP, and connect directly to that exact IP. Never re-resolve at connection time. The same principle applies to redirects: validate the new target before following it, or attackers just redirect their way around your check.
Webhook delivery looks like "just send an HTTP request" until you actually have to do it safely and at scale.
I'd love to hear how others have handled the slow-client / head-of-line-blocking problem. Always curious if there's a cleaner pattern than circuit breakers + tenant isolation.