Golang / GoLang Production Patterns and Web Standards Interview Questions
How do you implement load shedding and request queue limits in a Go HTTP server?
Load shedding protects a service from cascading failure under overload: when the system cannot keep up, it deliberately rejects additional requests rather than slowing down and failing all requests. Go implements this through semaphores and queue limits on the HTTP layer.
// Concurrency limiter: max N requests processed simultaneously type concurrencyLimiter struct { sem chan struct{} } func newConcurrencyLimiter(max int) *concurrencyLimiter { return &concurrencyLimiter{sem: make(chan struct{}, max)} } func (l *concurrencyLimiter) Middleware(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { select { case l.sem <- struct{}{}: // acquire slot default: // Immediately reject if at capacity â no waiting w.Header().Set("Retry-After", "1") http.Error(w, "server busy â try again later", http.StatusServiceUnavailable) return } defer func() { <-l.sem }() // release slot next.ServeHTTP(w, r) }) } // Request queue with timeout type queuedLimiter struct { queue chan struct{} timeout time.Duration } func (l *queuedLimiter) Middleware(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { ctx, cancel := context.WithTimeout(r.Context(), l.timeout) defer cancel() select { case l.queue <- struct{}{}: // wait up to timeout for a slot case <-ctx.Done(): http.Error(w, "request queue full", http.StatusServiceUnavailable) return } defer func() { <-l.queue }() next.ServeHTTP(w, r) }) } // Wire up limiter := newConcurrencyLimiter(100) // max 100 concurrent requests handler := limiter.Middleware(rateLimiter.Middleware(mux))
More Related questions...