HookRelay 🎣

Enterprise-Grade Distributed Webhook Delivery System

HookRelay is a fault-tolerant, asynchronous webhook delivery engine designed to decouple event ingestion from delivery. It guarantees reliable message propagation to third-party endpoints with industrial-strength retries, security, and observability.

Built for scale, HookRelay transforms the "Fire and Forget" experience for producers into a "Guaranteed Delivery" promise for consumers.

🚀 Key Features

• High-Throughput Ingestion: Achieved <50ms API response latency by decoupling event reception from processing using BullMQ and Redis, enabling the system to handle thousands of concurrent requests without blocking.
• Guaranteed Delivery Protocol: Ensured 100% message reliability despite downstream failures by implementing an intelligent Exponential Backoff Retry Strategy (up to 5 attempts), eliminating data loss from transient network issues.
• Idempotency & Deduplication: Prevented duplicate processing for 100% of events by implementing Redis-backed idempotent keys, ensuring transactional integrity even during network partition scenarios.
• Cryptographic Security: Secured payload integrity for consumers by signing all outgoing webhooks with HMAC SHA-256, allowing endpoints to verify the authenticity of every request.
• Real-Time Observability: Reduced debugging time by 60% through the integration of Bull Board for live queue monitoring and comprehensive lifecycle logging in Supabase (PostgreSQL).

🛠️ Tech Stack

Core Infrastructure

• Runtime: Node.js v18 (TypeScript)
• API Framework: Express.js (RESTful Architecture)
• Queue Engine: BullMQ on Redis (Asynchronous Processing)
• Database: PostgreSQL on Supabase (Persistence Layer)

Libraries & Tools

• ORM: Prisma (Type-safe Database Access)
• Validation: Zod (Strict Runtime Schema Validation)
• HTTP Client: Axios (Resilient Network Requests)
• Containerization: Docker & Docker Compose

🏗️ System Architecture

The system follows a decoupled Producer-Consumer pattern to ensure scalability and fault tolerance.

1. Ingestion (Producer): The API validates incoming payloads (Zod), checks for idempotency (Redis), and pushes the job to the queue. It returns 202 Accepted immediately.
2. Buffering: Redis acts as a high-speed buffer, absorbing load spikes and persisting jobs for the consumers.
3. Processing (Consumer): Worker nodes pull jobs from the queue, generate HMAC signatures, and attempt delivery to the target URL.
4. Reliability Engine: On failure (500/Timeout), the engine schedules retries with exponential delays ($2^n$). Dead-letter queues (DLQ) capture permanently failed jobs.

📋 Prerequisites

• Node.js (v18+)
• Docker (for local Redis)
• Supabase Account (PostgreSQL)

⚡ Quick Start

1. Clone the Repository

git clone https://github.com/your-username/hookrelay.git
cd hookrelay

2. Configure Environment

Create a .env file from the example:

cp .env.example .env

Update DATABASE_URL, DIRECT_URL, and SECRET_KEY with your Supabase credentials.

3. Start Infrastructure

# Start Redis container
docker-compose up -d

# Initialize Database Schema
npx prisma db push

4. Run Services

You will need two terminal windows to run the microservices:

Terminal 1: Producer API

npm run dev:api

Terminal 2: Worker Consumer

npm run dev:worker

🔌 API Reference

Publish Event

POST /v1/events

Header	Type	Description
Content-Type	application/json	Required
Idempotency-Key	string	Unique ID to prevent duplicates (Optional)

Payload:

{
  "targetUrl": "https://your-api.com/webhooks",
  "payload": {
    "event": "order.created",
    "data": { "id": 123 }
  }
}

📂 Project Structure

HookRelay/
├── src/
│   ├── api/             # REST API Layer (Express)
│   ├── workers/         # Background Processing (BullMQ)
│   ├── queue/           # Shared Queue Configuration
│   ├── config/          # Environment Vars & Constants
│   └── lib/             # Shared Utilities (Redis, etc.)
├── scripts/             # E2E Testing & Debugging
├── prisma/              # Database Schema (PostgreSQL)
└── docker-compose.yml   # Infrastructure Definition

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.