How do you implement load shedding and request queue limits in a Go HTTP server?

Load shedding protects a service from cascading failure under overload: when the system cannot keep up, it deliberately rejects additional requests rather than slowing down and failing all requests. Go implements this through semaphores and queue limits on the HTTP layer.

// Concurrency limiter: max N requests processed simultaneously
type concurrencyLimiter struct {
    sem chan struct{}
}

func newConcurrencyLimiter(max int) *concurrencyLimiter {
    return &concurrencyLimiter{sem: make(chan struct{}, max)}
}

func (l *concurrencyLimiter) Middleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        select {
        case l.sem <- struct{}{}: // acquire slot
        default:
            // Immediately reject if at capacity — no waiting
            w.Header().Set("Retry-After", "1")
            http.Error(w, "server busy — try again later",
                http.StatusServiceUnavailable)
            return
        }
        defer func() { <-l.sem }() // release slot
        next.ServeHTTP(w, r)
    })
}

// Request queue with timeout
type queuedLimiter struct {
    queue   chan struct{}
    timeout time.Duration
}

func (l *queuedLimiter) Middleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ctx, cancel := context.WithTimeout(r.Context(), l.timeout)
        defer cancel()

        select {
        case l.queue <- struct{}{}: // wait up to timeout for a slot
        case <-ctx.Done():
            http.Error(w, "request queue full",
                http.StatusServiceUnavailable)
            return
        }
        defer func() { <-l.queue }()
        next.ServeHTTP(w, r)
    })
}

// Wire up
limiter := newConcurrencyLimiter(100) // max 100 concurrent requests
handler := limiter.Middleware(rateLimiter.Middleware(mux))

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.