High-Performance API Gateway Rate Limiting with Redis

The Context: "The 3 AM Trigger"

It was late November 2023. We were hosting an open API for a high-frequency weather data provider. Suddenly, a single downstream client pushed a buggy script that spun up thousands of parallel threads making unprotected HTTP GET requests to our /v1/forecast/realtime endpoint. Our Spring Cloud Gateway, sitting at the edge, had a naive token-bucket rate limiter that relied on the gateway's in-memory state. Because we scaled the gateway pods horizontally (to 20 pods) to deal with the spike, the client essentially enjoyed 20x their intended rate limit. The backend Oracle database was battered with millions of identical select queries, triggering an auto-scaling event that cost us $4,000 in an hour. Absolute baigan configuration. We needed a globally synchronized, high-performance rate limiter.

The Problem Statement

Our constraints going forward:

• Accuracy: The rate limit state must be globally synchronized across all 20+ gateway pods.
• Algorithm: Move from basic Token Bucket to a precise Sliding Window Log to prevent burst exploitation at the edge of minute boundaries.
• Latency: Rejecting or passing a request at the edge must take < 3ms.
• Fail-Open: If the rate-limiting infrastructure goes down, we drop rate-limiting temporarily to preserve critical traffic.

Visual Architecture

+-RESETFULLSCREEN

The Core Logic / Process

Data Flow

Every incoming HTTP request routes through a global GlobalFilter in Spring Cloud Gateway. The filter examines the X-API-Key or IP address to determine the tenant identifier. Instead of sending multiple GET, SET, and EXPIRE commands to Redis, the Gateway sends a single, pre-compiled Lua script via the EVALSHA command. The Lua script executes atomizer logic inside the Redis engine.

Concurrency & Locking

The Sliding Window Log algorithm in Redis uses a Sorted Set (ZSET). The Lua script works as follows:

1. ZREMRANGEBYSCORE: Removes all request timestamps older than the window (e.g., now - 60 seconds).
2. ZCARD: Counts the remaining members.
3. If count < limit: ZADD the current timestamp and set a TTL on the key. Return ALLOWED (1).
4. If count >= limit: Return BLOCKED (0).

Because Lua scripts execute atomically in a single thread within Redis, there are zero race conditions. A client making 5,000 parallel requests will strictly be cut off at exactly request 101.

Failure Modes

Redis is fast, but network partitions happen. If the Spring Boot lettuce client times out waiting for Redis (timeout set to 5ms), the filter catches the exception and immediately Fails-Open. This means the request is allowed through, and we increment a micrometer counter rate_limit.redis.failures. It's better to let some burst traffic through to the auto-scaled backend than drop valid VIP customer traffic because a Redis pod is restarting.

The "Baigan" Take (Expert Insights)

Look, everyone reads about Sliding Window algorithms and tries to implement the ZSET logic directly in Java. They pull the set from Redis, modify it in the JVM, and write it back. This immediately introduces a massive Check-and-Set (CAS) race condition when 20 gateway pods are hitting the exact same key.

You absolutely must push the logic into the database. Pipelining or transactions (MULTI/EXEC) are not enough for this specific race condition. Using a Lua script ensures the entire evaluation and increment of the sliding window happens inside the Redis C engine atomically. It dropped our rate-limiter evaluation overhead from 15ms per request to 1.5ms.

Performance Checklist

• Use Redis Lua scripts (EVALSHA) to ensure atomic execution of the rate-limit check.
• Structure the Lua script to return exact remaining_tokens to populate the X-RateLimit-Remaining HTTP header.
• Implement a strict 5ms connection and read timeout on your Redis client.
• Wrap the evaluation in a try/catch and default to Fail-Open (allow request) during Redis outages.