Webhooks Explained: Complete Guide to HTTP Callbacks in 2025
In modern software development, real-time data synchronization between applications is essential. Whether you're processing payments with Stripe, automating deployments with GitHub, or sending notifications through Slack, webhooks power the event-driven architecture that makes these integrations possible.
This comprehensive guide covers everything you need to know about webhooks: how they work, when to use them, how to implement secure webhook endpoints, and best practices for production systems. By the end, you'll understand why webhooks have become the standard for real-time application integration.
What Are Webhooks?
Webhooks are automated HTTP callbacks that allow applications to send real-time data to other applications when specific events occur. Think of them as "reverse APIs" or "HTTP push notifications" for servers.
The Doorbell Analogy
The best way to understand webhooks is through the doorbell analogy:
Traditional API (Polling):
- You check your mailbox every hour to see if mail has arrived
- Most of the time, the mailbox is empty (wasted effort)
- If mail arrives at 9:05 AM, you won't see it until your 10:00 AM check (delayed notification)
Webhooks (Event-Driven):
- You install a doorbell on your mailbox
- The mail carrier rings the doorbell when delivering mail (instant notification)
- You only walk to the mailbox when mail actually arrives (efficient)
This fundamental difference makes webhooks dramatically more efficient than polling for real-time data synchronization.
Real-World Example
Let's say you run an e-commerce store using Stripe for payments. Here's what happens with webhooks:
- Customer completes checkout on your website
- Stripe processes the payment
- Stripe sends a webhook to your server: "Payment succeeded for Order #12345"
- Your server receives the webhook and updates the order status
- Your server sends a confirmation email to the customer
- Your server updates inventory
- All of this happens instantly and automatically
Without webhooks, you'd need to repeatedly ask Stripe "Any new payments? Any new payments?" every few seconds, which is inefficient and creates delays.
How Webhooks Work
Understanding the webhook workflow is essential for implementing them correctly.
The Webhook Lifecycle
┌─────────────────┐
│ Source App │
│ (e.g., Stripe) │
└────────┬────────┘
│
│ 1. Event occurs
│ (payment.succeeded)
│
▼
┌─────────────────┐
│ Webhook System │
│ - Builds payload│
│ - Signs payload │
│ - Sends POST │
└────────┬────────┘
│
│ 2. HTTP POST request
│ with JSON payload
│
▼
┌─────────────────┐
│ Your Server │
│ webhook.site │
└────────┬────────┘
│
│ 3. Verify signature
│ 4. Process event
│ 5. Return 200 OK
│
▼
┌─────────────────┐
│ Your App │
│ - Update DB │
│ - Send email │
│ - Trigger jobs │
└─────────────────┘
Step-by-Step Breakdown
1. Event Occurs in Source Application Something happens that triggers a webhook:
- Payment completed (Stripe)
- Code pushed to repository (GitHub)
- Message posted in channel (Discord)
- Support ticket created (Zendesk)
- Email delivered (SendGrid)
2. Webhook System Prepares Delivery The source application:
- Builds a JSON payload with event data
- Generates a cryptographic signature (for security)
- Looks up your registered webhook URL
- Sends an HTTP POST request to your endpoint
3. Your Server Receives the Webhook Your webhook endpoint:
- Receives the HTTP POST request
- Extracts the signature from headers
- Verifies the signature matches (authentication)
- Parses the JSON payload
- Returns a 200 OK response immediately
4. Asynchronous Processing After acknowledging receipt:
- Queue the webhook for background processing
- Update database records
- Send notifications
- Trigger additional workflows
- Log the event for debugging
5. Retry Mechanism (If Needed) If your server doesn't respond with 200:
- The webhook provider retries delivery
- Typically uses exponential backoff (1min, 5min, 15min, 1hr...)
- May retry for 24-72 hours depending on provider
- Eventually marks the webhook as failed
Common Use Cases for Webhooks
Webhooks power integrations across every industry. Here are the most common use cases:
1. Payment Processing
Providers: Stripe, PayPal, Square, Braintree
Events:
payment.succeeded- Process order fulfillmentpayment.failed- Notify customer, retry logicsubscription.canceled- Revoke accessrefund.created- Update accounting records
Why Webhooks Matter: Payment confirmations must be instant and reliable. Polling could miss critical events or introduce delays that affect customer experience.
2. CI/CD Automation
Providers: GitHub, GitLab, Bitbucket, Jenkins
Events:
push- Trigger automated tests and buildspull_request.opened- Run code quality checksrelease.published- Deploy to productionissue.labeled- Automated project management
Why Webhooks Matter: Developers expect immediate feedback on code changes. Webhooks enable real-time CI/CD pipelines that run tests and deployments automatically.
3. Team Communication
Providers: Slack, Discord, Microsoft Teams, Telegram
Events:
message.posted- Trigger bot responsesreaction.added- Track engagementchannel.created- Automated onboardingmember.joined- Welcome messages
Why Webhooks Matter: Real-time communication platforms require instant message delivery and bot responses. Webhooks enable interactive experiences that feel native to the platform.
4. Customer Support
Providers: Zendesk, Intercom, Help Scout, Freshdesk
Events:
ticket.created- Auto-assign to team membersticket.updated- Notify stakeholderssatisfaction.rated- Track support qualityagent.replied- Update external systems
Why Webhooks Matter: Support teams need instant notifications about new tickets and customer responses to maintain fast response times.
5. Email Marketing
Providers: SendGrid, Mailchimp, Postmark, Mailgun
Events:
email.delivered- Confirm successful deliveryemail.opened- Track engagementemail.clicked- Measure campaign performanceemail.bounced- Clean email listsemail.spam_report- Maintain sender reputation
Why Webhooks Matter: Email deliverability and engagement tracking require real-time data to optimize campaigns and maintain sender reputation.
6. Monitoring & Alerts
Providers: Datadog, PagerDuty, Sentry, New Relic
Events:
alert.triggered- Incident response automationmetric.threshold_exceeded- Scale infrastructureerror.occurred- Developer notificationsservice.down- Automated failover
Why Webhooks Matter: System outages and errors require immediate response. Webhooks enable automated incident response and alert escalation.
7. E-Commerce & Inventory
Providers: Shopify, WooCommerce, BigCommerce, Magento
Events:
order.created- Trigger fulfillmentinventory.low- Reorder stockcustomer.created- Add to CRMproduct.updated- Sync to marketplace
Why Webhooks Matter: E-commerce operations require real-time inventory tracking and order processing to prevent overselling and ensure timely fulfillment.
8. Project Management
Providers: Jira, Linear, Asana, Monday.com
Events:
issue.created- Notify team membersissue.status_changed- Update dashboardscomment.added- Real-time collaborationproject.completed- Trigger celebrations 🎉
Why Webhooks Matter: Modern project management tools require real-time updates so teams can collaborate effectively without refreshing pages.
Webhook Security: Authentication & Verification
Security is critical for webhook implementations. Without proper verification, attackers could send malicious webhooks to your endpoint, potentially triggering fraudulent orders, data corruption, or unauthorized actions.
Why Signature Verification Matters
Consider this attack scenario:
# Attacker sends fake webhook to your endpoint
curl -X POST https://yoursite.com/webhooks/stripe \
-H "Content-Type: application/json" \
-d '{
"type": "payment_intent.succeeded",
"data": {
"object": {
"id": "pi_fake123",
"amount": 10000,
"customer": "cus_attacker"
}
}
}'
Without signature verification, your server would:
- Receive this fake webhook
- Assume it's legitimate
- Grant access to paid features
- Fulfill a fraudulent order
- Lose money and inventory
With signature verification, your server:
- Receives the webhook
- Computes expected signature using your secret key
- Compares to the signature in headers
- Rejects the request because signatures don't match
- Prevents the attack
Common Signature Algorithms
Different webhook providers use different signature algorithms:
HMAC-SHA256 (Most Common)
Used by: Stripe, GitHub, Slack, Shopify, Zendesk, Linear
Header: X-Provider-Signature (varies by provider)
Algorithm:
const crypto = require('crypto');
function verifyWebhook(payload, signature, secret) {
const expectedSignature = crypto
.createHmac('sha256', secret)
.update(payload)
.digest('hex');
return crypto.timingSafeEqual(
Buffer.from(signature),
Buffer.from(expectedSignature)
);
}
Example (Stripe):
const stripe = require('stripe')(process.env.STRIPE_SECRET_KEY);
app.post('/webhooks/stripe', express.raw({type: 'application/json'}), (req, res) => {
const signature = req.headers['stripe-signature'];
const secret = process.env.STRIPE_WEBHOOK_SECRET;
let event;
try {
// Stripe SDK handles verification
event = stripe.webhooks.constructEvent(req.body, signature, secret);
} catch (err) {
console.error('Webhook signature verification failed:', err.message);
return res.status(400).send(`Webhook Error: ${err.message}`);
}
// Webhook verified, process event
switch (event.type) {
case 'payment_intent.succeeded':
const paymentIntent = event.data.object;
console.log('Payment succeeded:', paymentIntent.id);
break;
// Handle other events...
}
res.json({received: true});
});
HMAC-SHA1
Used by: Twilio
Algorithm:
function verifyTwilioWebhook(url, params, signature, authToken) {
// Twilio signs: URL + sorted POST params
const data = url + Object.keys(params).sort().map(key => key + params[key]).join('');
const expectedSignature = crypto
.createHmac('sha1', authToken)
.update(Buffer.from(data, 'utf-8'))
.digest('base64');
return expectedSignature === signature;
}
RSA-SHA256
Used by: PayPal
Algorithm:
const crypto = require('crypto');
function verifyPayPalWebhook(payload, signature, certUrl) {
// 1. Fetch PayPal's public certificate
const cert = fetchCertificate(certUrl);
// 2. Verify signature with public key
const verifier = crypto.createVerify('RSA-SHA256');
verifier.update(payload);
return verifier.verify(cert, signature, 'base64');
}
ECDSA (Elliptic Curve)
Used by: SendGrid
Algorithm:
const crypto = require('crypto');
function verifySendGridWebhook(payload, signature, timestamp, publicKey) {
const timestampedPayload = timestamp + payload;
const verifier = crypto.createVerify('ecdsa-with-SHA256');
verifier.update(timestampedPayload);
return verifier.verify(publicKey, signature, 'base64');
}
No Signature
Used by: Discord (URL-based security), Jira (optional)
Security Approach:
- Use long, random, secret URLs
- Implement IP allowlisting
- Add your own authentication layer
- Not recommended for production systems
Timestamp Verification (Replay Attack Prevention)
Many providers include timestamps to prevent replay attacks:
function verifyTimestamp(timestamp, maxAge = 300) {
const now = Math.floor(Date.now() / 1000);
const age = now - timestamp;
if (age > maxAge) {
throw new Error('Webhook timestamp too old (possible replay attack)');
}
return true;
}
Example (Stripe):
const stripe = require('stripe')(process.env.STRIPE_SECRET_KEY);
// Stripe includes timestamp in signature
// Extract with: stripe.webhooks.constructEvent()
// Automatically rejects webhooks older than 5 minutes
Security Best Practices Checklist
- ✅ Always verify signatures when the provider supports it
- ✅ Use timing-safe comparison (
crypto.timingSafeEqual) to prevent timing attacks - ✅ Validate timestamps to prevent replay attacks (max age: 5 minutes)
- ✅ Use HTTPS endpoints only to prevent man-in-the-middle attacks
- ✅ Store secrets in environment variables, never in code
- ✅ Rotate webhook secrets periodically (every 90 days)
- ✅ Implement rate limiting to prevent abuse (e.g., 100 requests/minute per IP)
- ✅ Log all webhook attempts (successful and failed) for debugging
- ✅ Use raw body for signature verification (not parsed JSON)
- ✅ Return 200 immediately, process asynchronously to avoid timeouts
Testing Webhooks Locally
Testing webhooks during development requires exposing your localhost to the internet. Here's how to do it effectively.
Method 1: ngrok (Most Popular)
ngrok creates a secure tunnel to your localhost with a public HTTPS URL.
Setup:
# Install ngrok
brew install ngrok # macOS
# or download from https://ngrok.com
# Start your local server
node server.js # Running on localhost:3000
# Create tunnel
ngrok http 3000
Output:
Session Status online
Account [email protected]
Forwarding https://abc123.ngrok.io -> http://localhost:3000
Use the ngrok URL as your webhook endpoint:
https://abc123.ngrok.io/webhooks/stripe
View webhook requests in ngrok's web interface:
http://localhost:4040
Method 2: Cloudflare Tunnel (Free, No Account Required)
# Install cloudflared
brew install cloudflare/cloudflare/cloudflared
# Start tunnel
cloudflared tunnel --url http://localhost:3000
Method 3: localtunnel (Open Source)
# Install globally
npm install -g localtunnel
# Create tunnel
lt --port 3000
# Output: your url is: https://random-subdomain.loca.lt
Method 4: Webhook Testing Tools
Use our Webhook Payload Generator to:
- Generate realistic webhook payloads for any provider
- Test signature verification locally
- Simulate different event types
- Copy/paste payloads directly into your code
Example workflow:
- Visit Webhook Payload Generator
- Select "Stripe" provider
- Choose "payment_intent.succeeded" event
- Generate payload with signature
- Copy payload and send to your local endpoint:
curl -X POST http://localhost:3000/webhooks/stripe \
-H "Content-Type: application/json" \
-H "Stripe-Signature: t=1234567890,v1=abc123..." \
-d @webhook-payload.json
Debugging Webhook Requests
Log everything during development:
const express = require('express');
const app = express();
// Webhook endpoint with comprehensive logging
app.post('/webhooks/:provider',
express.raw({type: 'application/json'}),
(req, res) => {
console.log('=== Webhook Received ===');
console.log('Provider:', req.params.provider);
console.log('Headers:', JSON.stringify(req.headers, null, 2));
console.log('Body (raw):', req.body.toString());
console.log('Body (parsed):', JSON.parse(req.body.toString()));
console.log('========================');
res.json({received: true});
}
);
Common debugging issues:
| Issue | Cause | Solution |
|---|---|---|
| Signature verification fails | Body parsed before verification | Use express.raw() middleware |
| Request times out | Processing takes too long | Return 200 immediately, process async |
| Duplicate events | No idempotency check | Track event IDs in database |
| Missing headers | Tunneling tool modifies requests | Check tunnel configuration |
| SSL errors | Localhost using HTTP | Use ngrok for HTTPS tunnel |
Implementing Webhook Endpoints
Let's build a production-ready webhook endpoint from scratch.
Basic Webhook Endpoint (Node.js + Express)
const express = require('express');
const crypto = require('crypto');
require('dotenv').config();
const app = express();
// IMPORTANT: Use raw body for signature verification
app.use('/webhooks', express.raw({type: 'application/json'}));
app.post('/webhooks/stripe', (req, res) => {
// 1. Extract signature and secret
const signature = req.headers['stripe-signature'];
const secret = process.env.STRIPE_WEBHOOK_SECRET;
// 2. Verify signature
try {
const stripe = require('stripe')(process.env.STRIPE_SECRET_KEY);
const event = stripe.webhooks.constructEvent(req.body, signature, secret);
// 3. Return 200 immediately
res.status(200).json({received: true});
// 4. Process asynchronously
processStripeWebhook(event);
} catch (err) {
console.error('Webhook verification failed:', err.message);
return res.status(400).send(`Webhook Error: ${err.message}`);
}
});
function processStripeWebhook(event) {
switch (event.type) {
case 'payment_intent.succeeded':
console.log('Payment succeeded:', event.data.object.id);
// Update database, send confirmation email, etc.
break;
case 'customer.subscription.deleted':
console.log('Subscription canceled:', event.data.object.id);
// Revoke access, send cancellation email, etc.
break;
default:
console.log('Unhandled event type:', event.type);
}
}
app.listen(3000, () => console.log('Webhook server running on port 3000'));
Production Webhook Endpoint (with Queue)
For production systems, always use a queue to process webhooks asynchronously:
const express = require('express');
const crypto = require('crypto');
const Queue = require('bull');
const Redis = require('ioredis');
require('dotenv').config();
const app = express();
const redis = new Redis(process.env.REDIS_URL);
// Create webhook queue
const webhookQueue = new Queue('stripe-webhooks', {
redis: process.env.REDIS_URL
});
// Webhook endpoint
app.post('/webhooks/stripe',
express.raw({type: 'application/json'}),
async (req, res) => {
try {
// 1. Verify signature
const signature = req.headers['stripe-signature'];
const stripe = require('stripe')(process.env.STRIPE_SECRET_KEY);
const event = stripe.webhooks.constructEvent(
req.body,
signature,
process.env.STRIPE_WEBHOOK_SECRET
);
// 2. Check for duplicate (idempotency)
const eventId = event.id;
const exists = await redis.get(`webhook:${eventId}`);
if (exists) {
console.log('Duplicate webhook ignored:', eventId);
return res.status(200).json({received: true, duplicate: true});
}
// 3. Mark as received
await redis.setex(`webhook:${eventId}`, 86400, 'processing'); // 24 hour TTL
// 4. Add to queue
await webhookQueue.add({
eventId: eventId,
type: event.type,
data: event.data.object
}, {
attempts: 3,
backoff: {
type: 'exponential',
delay: 2000
}
});
// 5. Return 200 immediately
res.status(200).json({received: true, queued: true});
} catch (err) {
console.error('Webhook error:', err.message);
// Return 200 even on error to prevent retries for invalid signatures
res.status(200).json({received: true, error: true});
}
}
);
// Process webhooks from queue
webhookQueue.process(async (job) => {
const { eventId, type, data } = job.data;
console.log('Processing webhook:', eventId, type);
try {
switch (type) {
case 'payment_intent.succeeded':
await handlePaymentSuccess(data);
break;
case 'customer.subscription.deleted':
await handleSubscriptionCanceled(data);
break;
default:
console.log('Unhandled event type:', type);
}
// Mark as completed
await redis.setex(`webhook:${eventId}`, 86400, 'completed');
} catch (err) {
console.error('Webhook processing error:', err);
throw err; // Bull will retry based on job configuration
}
});
// Webhook processing functions
async function handlePaymentSuccess(paymentIntent) {
// Update database
await db.orders.update({
stripePaymentIntentId: paymentIntent.id
}, {
status: 'paid',
paidAt: new Date()
});
// Send confirmation email
await sendEmail({
to: paymentIntent.receipt_email,
subject: 'Payment Received',
template: 'payment-confirmation',
data: { amount: paymentIntent.amount / 100 }
});
}
async function handleSubscriptionCanceled(subscription) {
// Revoke access
await db.users.update({
stripeSubscriptionId: subscription.id
}, {
subscriptionStatus: 'canceled',
accessUntil: new Date(subscription.current_period_end * 1000)
});
// Send cancellation email
await sendEmail({
to: subscription.customer.email,
subject: 'Subscription Canceled',
template: 'subscription-canceled'
});
}
app.listen(3000);
Python + Flask Implementation
from flask import Flask, request, jsonify
import hmac
import hashlib
import json
import os
from redis import Redis
from rq import Queue
app = Flask(__name__)
redis_conn = Redis(host='localhost', port=6379)
queue = Queue('webhooks', connection=redis_conn)
@app.route('/webhooks/stripe', methods=['POST'])
def stripe_webhook():
# 1. Get signature and payload
signature = request.headers.get('Stripe-Signature')
payload = request.data
secret = os.environ.get('STRIPE_WEBHOOK_SECRET')
# 2. Verify signature
try:
event = verify_stripe_webhook(payload, signature, secret)
except Exception as e:
print(f'Webhook verification failed: {e}')
return jsonify({'error': str(e)}), 400
# 3. Check for duplicate
event_id = event['id']
if redis_conn.exists(f'webhook:{event_id}'):
print(f'Duplicate webhook ignored: {event_id}')
return jsonify({'received': True, 'duplicate': True}), 200
# 4. Mark as received
redis_conn.setex(f'webhook:{event_id}', 86400, 'processing')
# 5. Add to queue
queue.enqueue(process_stripe_webhook, event)
# 6. Return 200 immediately
return jsonify({'received': True, 'queued': True}), 200
def verify_stripe_webhook(payload, signature, secret):
# Extract timestamp and signatures
elements = signature.split(',')
timestamp = int([e.split('=')[1] for e in elements if e.startswith('t=')][0])
signatures = [e.split('=')[1] for e in elements if e.startswith('v1=')]
# Verify timestamp (prevent replay attacks)
import time
if abs(time.time() - timestamp) > 300: # 5 minutes
raise Exception('Webhook timestamp too old')
# Compute expected signature
signed_payload = f'{timestamp}.{payload.decode()}'
expected_signature = hmac.new(
secret.encode(),
signed_payload.encode(),
hashlib.sha256
).hexdigest()
# Verify signature
if expected_signature not in signatures:
raise Exception('Invalid signature')
# Return parsed event
return json.loads(payload)
def process_stripe_webhook(event):
event_type = event['type']
data = event['data']['object']
if event_type == 'payment_intent.succeeded':
handle_payment_success(data)
elif event_type == 'customer.subscription.deleted':
handle_subscription_canceled(data)
else:
print(f'Unhandled event type: {event_type}')
if __name__ == '__main__':
app.run(port=3000)
PHP Implementation
<?php
require 'vendor/autoload.php';
use Stripe\Stripe;
use Stripe\Webhook;
Stripe::setApiKey($_ENV['STRIPE_SECRET_KEY']);
// Get raw POST data
$payload = file_get_contents('php://input');
$signature = $_SERVER['HTTP_STRIPE_SIGNATURE'];
$secret = $_ENV['STRIPE_WEBHOOK_SECRET'];
try {
// Verify webhook signature
$event = Webhook::constructEvent($payload, $signature, $secret);
// Return 200 immediately
http_response_code(200);
echo json_encode(['received' => true]);
// Flush response to client
if (function_exists('fastcgi_finish_request')) {
fastcgi_finish_request();
}
// Process asynchronously
processStripeWebhook($event);
} catch (\UnexpectedValueException $e) {
// Invalid payload
http_response_code(400);
exit();
} catch (\Stripe\Exception\SignatureVerificationException $e) {
// Invalid signature
http_response_code(400);
exit();
}
function processStripeWebhook($event) {
switch ($event->type) {
case 'payment_intent.succeeded':
$paymentIntent = $event->data->object;
handlePaymentSuccess($paymentIntent);
break;
case 'customer.subscription.deleted':
$subscription = $event->data->object;
handleSubscriptionCanceled($subscription);
break;
default:
error_log('Unhandled event type: ' . $event->type);
}
}
?>
Debugging Webhooks
Common issues and how to fix them:
Issue 1: Signature Verification Fails
Symptoms:
Error: No signatures found matching the expected signature for payload
Causes:
- Body was parsed as JSON before verification
- Using wrong webhook secret
- Timestamp too old (replay attack protection)
Solution:
// ❌ Wrong - body parsed before verification
app.use(express.json());
app.post('/webhooks/stripe', (req, res) => {
// req.body is already parsed, signature verification will fail
});
// ✅ Correct - use raw body
app.use('/webhooks', express.raw({type: 'application/json'}));
app.post('/webhooks/stripe', (req, res) => {
// req.body is Buffer, signature verification works
});
Issue 2: Webhook Timeouts
Symptoms:
Webhook delivery failed: Timeout after 5000ms
Cause: Processing takes too long, webhook provider times out
Solution:
// ❌ Wrong - processing before responding
app.post('/webhooks/stripe', async (req, res) => {
const event = verifyWebhook(req);
await updateDatabase(event); // Takes 2 seconds
await sendEmail(event); // Takes 3 seconds
await triggerAnalytics(event); // Takes 1 second
res.json({received: true}); // Total: 6 seconds → TIMEOUT
});
// ✅ Correct - respond immediately, process async
app.post('/webhooks/stripe', async (req, res) => {
const event = verifyWebhook(req);
res.json({received: true}); // Respond in < 100ms
// Process asynchronously
await webhookQueue.add(event);
});
Issue 3: Duplicate Webhook Processing
Symptoms:
- Orders fulfilled twice
- Emails sent twice
- Database conflicts
Cause: Webhook provider retries due to slow response or error
Solution:
// Implement idempotency with event ID tracking
const processedEvents = new Set();
app.post('/webhooks/stripe', async (req, res) => {
const event = verifyWebhook(req);
// Check if already processed
if (processedEvents.has(event.id)) {
console.log('Duplicate webhook ignored:', event.id);
return res.json({received: true, duplicate: true});
}
// Mark as processed
processedEvents.add(event.id);
res.json({received: true});
await webhookQueue.add(event);
});
// Better: Use database or Redis for persistence
const redis = require('redis').createClient();
app.post('/webhooks/stripe', async (req, res) => {
const event = verifyWebhook(req);
const exists = await redis.get(`webhook:${event.id}`);
if (exists) {
return res.json({received: true, duplicate: true});
}
await redis.setex(`webhook:${event.id}`, 86400, 'processed'); // 24h TTL
res.json({received: true});
await webhookQueue.add(event);
});
Issue 4: Webhooks Stopped Being Delivered
Symptoms:
- Webhooks worked yesterday, now nothing is received
- No requests hitting your endpoint
Common Causes:
- SSL certificate expired - Webhook providers require HTTPS with valid certificates
- Endpoint URL changed - Deployment changed your webhook URL
- Too many failures - Provider disabled webhooks after repeated failures
- IP address changed - If using IP allowlisting
Solution:
- Check webhook configuration in provider dashboard
- Verify your endpoint is accessible:
curl -I https://yoursite.com/webhooks/stripe - Check webhook delivery logs in provider dashboard
- Test with manual webhook trigger (most providers support this)
- Re-enable webhooks if disabled due to failures
Debugging Checklist
Use this checklist when webhooks aren't working:
- Verify endpoint is accessible -
curl https://yoursite.com/webhooks/provider - Check SSL certificate - Valid HTTPS required
- Review webhook configuration - Correct URL in provider dashboard
- Verify webhook secret - Using correct environment variable
- Check event types - Subscribed to the events you expect
- Review request logs - Check server logs for incoming requests
- Test signature verification - Use provider's test events
- Verify timestamp tolerance - Check if timestamp validation too strict
- Check rate limits - Ensure not hitting rate limits
- Review error logs - Look for exceptions in processing code
- Test locally with tunnel - Use ngrok to test in development
- Check queue processing - Verify background jobs are running
- Monitor delivery stats - Use provider dashboard to see delivery success rate
Webhook Best Practices
Follow these best practices for production webhook implementations:
1. Always Return 200 Immediately
Why: Webhook providers expect quick responses. If you take too long, they'll retry, causing duplicates.
// ❌ Bad - slow response
app.post('/webhooks/stripe', async (req, res) => {
const event = verifyWebhook(req);
await processEvent(event); // Takes 5 seconds
res.json({received: true}); // Response delayed
});
// ✅ Good - fast response
app.post('/webhooks/stripe', async (req, res) => {
const event = verifyWebhook(req);
res.json({received: true}); // Response in < 100ms
await queue.add(event); // Process asynchronously
});
2. Process Webhooks Asynchronously with Queues
Use a queue system (Bull, Redis, RabbitMQ, AWS SQS) for processing:
const Queue = require('bull');
const webhookQueue = new Queue('webhooks');
// Endpoint adds to queue
app.post('/webhooks/stripe', async (req, res) => {
const event = verifyWebhook(req);
await webhookQueue.add(event);
res.json({received: true});
});
// Worker processes from queue
webhookQueue.process(async (job) => {
await processStripeEvent(job.data);
});
Benefits:
- Fast webhook responses (< 100ms)
- Automatic retries with exponential backoff
- Horizontal scaling (add more workers)
- Failed job tracking and dead-letter queues
- Monitoring and observability
3. Implement Idempotency (Duplicate Detection)
Track event IDs to prevent duplicate processing:
// Using Redis for distributed systems
const redis = require('redis').createClient();
async function processWebhook(event) {
const key = `webhook:${event.id}`;
// Check if already processed
const exists = await redis.get(key);
if (exists) {
console.log('Duplicate ignored:', event.id);
return;
}
// Mark as processing
await redis.setex(key, 86400, 'processing'); // 24 hour TTL
try {
// Process event
await handleEvent(event);
// Mark as completed
await redis.set(key, 'completed');
} catch (err) {
// Mark as failed
await redis.set(key, 'failed');
throw err;
}
}
4. Log Everything (But Redact Sensitive Data)
Comprehensive logging is essential for debugging:
const winston = require('winston');
const logger = winston.createLogger({
level: 'info',
format: winston.format.json(),
transports: [
new winston.transports.File({ filename: 'webhooks.log' })
]
});
app.post('/webhooks/stripe', async (req, res) => {
const startTime = Date.now();
const event = verifyWebhook(req);
logger.info('Webhook received', {
eventId: event.id,
eventType: event.type,
timestamp: new Date().toISOString(),
ip: req.ip,
userAgent: req.get('user-agent')
});
res.json({received: true});
try {
await queue.add(event);
logger.info('Webhook queued', {
eventId: event.id,
duration: Date.now() - startTime
});
} catch (err) {
logger.error('Webhook queue error', {
eventId: event.id,
error: err.message,
stack: err.stack
});
}
});
// Redact sensitive data
function sanitizeEvent(event) {
const sanitized = JSON.parse(JSON.stringify(event));
// Remove sensitive fields
if (sanitized.data?.object?.card) {
sanitized.data.object.card.number = '****';
sanitized.data.object.card.cvc = '***';
}
return sanitized;
}
5. Implement Monitoring and Alerts
Track webhook health with metrics:
const prometheus = require('prom-client');
// Create metrics
const webhookReceived = new prometheus.Counter({
name: 'webhooks_received_total',
help: 'Total webhooks received',
labelNames: ['provider', 'event_type']
});
const webhookProcessed = new prometheus.Counter({
name: 'webhooks_processed_total',
help: 'Total webhooks processed',
labelNames: ['provider', 'event_type', 'status']
});
const webhookDuration = new prometheus.Histogram({
name: 'webhook_processing_duration_seconds',
help: 'Webhook processing duration',
labelNames: ['provider', 'event_type']
});
// Instrument webhook endpoint
app.post('/webhooks/stripe', async (req, res) => {
const event = verifyWebhook(req);
webhookReceived.inc({
provider: 'stripe',
event_type: event.type
});
res.json({received: true});
const timer = webhookDuration.startTimer({
provider: 'stripe',
event_type: event.type
});
try {
await processEvent(event);
webhookProcessed.inc({
provider: 'stripe',
event_type: event.type,
status: 'success'
});
} catch (err) {
webhookProcessed.inc({
provider: 'stripe',
event_type: event.type,
status: 'error'
});
} finally {
timer();
}
});
Set up alerts for:
- Webhook failure rate > 5%
- Processing time > 10 seconds
- Queue depth > 1000 jobs
- No webhooks received in 1 hour (for high-volume systems)
6. Handle Webhook Provider Retries Gracefully
Most webhook providers implement retry logic:
| Provider | Retry Duration | Retry Schedule |
|---|---|---|
| Stripe | 3 days | 1h, 2h, 4h, 8h, 16h, 24h (exponential) |
| GitHub | 24 hours | 1m, 5m, 15m, 30m, 1h, 3h, 6h, 12h, 24h |
| Discord | 15 minutes | Exponential backoff |
| Shopify | 48 hours | 19 attempts over 48 hours |
| Twilio | 4 hours | Exponential backoff |
Best practices:
- Return 200 even if processing fails (prevents unnecessary retries)
- Use idempotency to handle duplicate deliveries
- Don't retry on your side if provider is already retrying
- Monitor retry patterns to identify systemic issues
7. Use Webhook Validation in Development
Test with provider SDKs that validate payloads:
const Joi = require('joi');
// Define schema for expected webhook structure
const stripePaymentIntentSchema = Joi.object({
id: Joi.string().required(),
object: Joi.string().valid('payment_intent').required(),
amount: Joi.number().integer().min(0).required(),
currency: Joi.string().length(3).required(),
status: Joi.string().valid('succeeded', 'canceled', 'processing').required()
});
function processPaymentIntent(paymentIntent) {
// Validate structure
const { error, value } = stripePaymentIntentSchema.validate(paymentIntent);
if (error) {
console.error('Invalid payment intent structure:', error.details);
throw new Error('Invalid webhook payload');
}
// Process validated data
console.log('Processing payment:', value.id);
}
8. Implement Dead Letter Queues
Capture failed webhooks for manual review:
const webhookQueue = new Queue('webhooks');
const deadLetterQueue = new Queue('webhooks-dlq');
webhookQueue.process(async (job) => {
try {
await processEvent(job.data);
} catch (err) {
// After max retries, move to DLQ
if (job.attemptsMade >= 3) {
await deadLetterQueue.add({
originalJob: job.data,
error: err.message,
attempts: job.attemptsMade,
failedAt: new Date()
});
console.error('Webhook moved to DLQ:', job.data.id);
}
throw err; // Bull will retry
}
});
// Manual review process
deadLetterQueue.process(async (job) => {
// Alert team for manual intervention
await sendAlert({
title: 'Webhook Failed After Retries',
data: job.data,
severity: 'high'
});
});
Popular Webhook Providers
Here's a comprehensive guide to webhook providers across different categories:
Payment Processing
- Stripe Webhooks - Payment events, subscriptions, refunds
- PayPal Webhooks - Payment notifications, disputes, subscriptions
- Square Webhooks - Point-of-sale, payments, inventory
Development & Version Control
- GitHub Webhooks - Push, PR, issues, releases
- GitLab Webhooks - Pipeline events, merge requests, deployments
- Bitbucket Webhooks - Repository events, build status
Communication
- Slack Webhooks - Messages, reactions, channel events
- Discord Webhooks - Messages, embeds, bot interactions
- Microsoft Teams Webhooks - Channel notifications, adaptive cards
- Twilio Webhooks - SMS, voice calls, messaging
- SendGrid Webhooks - Delivery, opens, clicks, bounces
- Mailchimp Webhooks - Subscribe, unsubscribe, campaign events
- Postmark Webhooks - Delivery tracking, bounce handling
- Mailgun Webhooks - Email events, spam complaints
Customer Support
- Zendesk Webhooks - Ticket events, comments, status changes
- Intercom Webhooks - Conversations, user events, lead qualification
- Help Scout Webhooks - Ticket creation, customer events
Project Management
- Jira Webhooks - Issue events, sprint changes, comments
- Linear Webhooks - Issue updates, project changes
- Asana Webhooks - Task changes, project updates
- Monday.com Webhooks - Board updates, item changes
E-Commerce
- Shopify Webhooks - Orders, products, customers, inventory
- WooCommerce Webhooks - Order events, product changes
- BigCommerce Webhooks - Store events, cart operations
CRM & Marketing
- HubSpot Webhooks - Contacts, deals, companies
- Salesforce Webhooks - CRM events, opportunities, leads
Monitoring & Observability
- Datadog Webhooks - Alert notifications, metric thresholds
- PagerDuty Webhooks - Incidents, escalations, acknowledgments
- Sentry Webhooks - Error events, release tracking
- New Relic Webhooks - Performance alerts, deployments
Each provider has unique implementation details, signature algorithms, and event types. Visit our provider-specific guides above for complete implementation examples, code samples, and troubleshooting tips.
Tools & Resources
Our Free Tools
- Generate realistic webhook payloads for 14+ providers
- Test signature verification locally
- Copy/paste ready-to-use JSON
- Supports Stripe, GitHub, Slack, Discord, Twilio, SendGrid, and more
- Privacy-first: 100% client-side processing
- Pretty-print webhook JSON payloads
- Validate JSON structure
- Syntax highlighting
- Build and test HTTP requests
- Custom headers and authentication
- Test webhook endpoints
Webhook Testing Services
webhook.site - Instantly test webhooks with unique URLs requestbin.com - Inspect HTTP requests in real-time ngrok.com - Secure tunnels to localhost for testing localtunnel.me - Alternative tunneling solution
Provider Documentation
- Stripe Webhooks Documentation
- GitHub Webhooks Documentation
- Slack API Events
- Discord Webhooks Guide
- Twilio Webhooks Security
- SendGrid Event Webhook
- Shopify Webhooks
Libraries & SDKs
Node.js:
stripe- Official Stripe SDK with webhook utilities@octokit/webhooks- GitHub Webhooks SDK@slack/bolt- Slack Bolt frameworksvix- Webhook infrastructure library
Python:
stripe- Official Stripe SDKgithub- PyGithub libraryslack_sdk- Official Slack SDKsvix- Webhook infrastructure library
PHP:
stripe/stripe-php- Official Stripe SDKlaravel/cashier-stripe- Laravel Stripe integrationsymfony/webhook- Symfony Webhook component
Queue & Background Processing
Node.js:
bull- Redis-based queue (recommended)bee-queue- Simple Redis queueaws-sdk- AWS SQSamqplib- RabbitMQ client
Python:
celery- Distributed task queue (recommended)rq- Redis Queuedramatiq- Alternative to Celery
PHP:
- Laravel Queues (built-in)
- Symfony Messenger
enqueue/enqueue- Universal queue library
Conclusion
Webhooks are the backbone of modern application integrations, enabling real-time, event-driven architectures that power everything from payment processing to CI/CD automation. By understanding how webhooks work, implementing proper security with signature verification, and following production best practices like asynchronous processing and idempotency, you can build reliable webhook integrations that scale.
Key Takeaways
✅ Webhooks are event-driven HTTP callbacks that push data in real-time, eliminating the need for constant polling
✅ Always verify signatures using HMAC-SHA256 (or provider-specific algorithm) to prevent malicious webhook attacks
✅ Return 200 immediately and process webhooks asynchronously using queues to avoid timeouts and retries
✅ Implement idempotency with event ID tracking to handle duplicate deliveries gracefully
✅ Test locally with ngrok or similar tunneling tools, and use our Webhook Payload Generator for realistic test data
✅ Monitor webhook health with metrics, alerts, and dead-letter queues for failed events
✅ Use provider SDKs when available for built-in signature verification and type-safe webhook handling
Next Steps
- Choose your provider - Select from our provider-specific guides for detailed implementation instructions
- Test locally - Use ngrok and our Webhook Payload Generator to test your implementation
- Implement security - Add signature verification before deploying to production
- Set up monitoring - Track webhook delivery and processing metrics
- Scale with queues - Implement asynchronous processing with Bull, Celery, or your platform's queue system
Ready to start building? Visit our Webhook Payload Generator to generate test payloads for any provider and start implementing webhook endpoints today.
Have questions about webhooks? Check out our provider-specific guides, troubleshooting articles, or contact our team for expert guidance on webhook implementations.
Related Articles:
- Discord Webhooks: Complete Guide with Examples
- Stripe Webhooks: Payment Event Handling
- Testing Webhooks Locally with ngrok
- Webhook Signature Verification Best Practices
Related Tools:
- Webhook Payload Generator - Generate realistic webhook payloads
- JSON Formatter - Format and validate webhook JSON
- API Request Builder - Test webhook endpoints