Spring MVC REST APIs: Bounded Concurrency, Deadlines, and Context

Spring MVC and spring rest api: define an end-to-end deadline

A request timeout on the ingress proxy is not an end-to-end budget. In a microservices architecture, a 2-second gateway timeout followed by three downstream clients with independent 2-second read timeouts can consume six seconds of work after the caller has disconnected. A practical spring framework training exercise is to accept an absolute X-Request-Deadline timestamp, reject expired requests immediately, and forward the same deadline rather than resetting a relative timeout at every hop. This is equally useful in spring boot training because it forces teams to reason about the whole request graph, not a single controller.

@Component
final class DeadlineInterceptor implements HandlerInterceptor {
  private final Clock clock;

  DeadlineInterceptor(Clock clock) { this.clock = clock; }

  @Override
  public boolean preHandle(HttpServletRequest request,
                           HttpServletResponse response,
                           Object handler) {
    long fallback = clock.millis() + 2_500;
    long deadline = Optional.ofNullable(request.getHeader("X-Request-Deadline"))
        .map(Long::parseLong)
        .orElse(fallback);

    if (deadline <= clock.millis()) {
      response.setStatus(504);
      return false;
    }
    request.setAttribute("deadlineEpochMs", deadline);
    return true;
  }
}

Before each downstream call, calculate remainingMs = deadline - clock.millis() and fail locally when it is below a safety margin such as 50 ms. Send X-Request-Deadline downstream and emit a Micrometer timer tagged only with bounded values such as dependency=catalog and outcome=deadline_exhausted. Do not tag a timer with a request ID, user ID, URL path containing an ID, or raw exception message: those values create unbounded time-series cardinality in Prometheus.

Size HTTP connection pools before enabling more Spring MVC concurrency

Virtual threads can make blocking servlet code easier to scale, but they do not create downstream database or HTTP capacity. If 1,000 request tasks simultaneously wait for a dependency backed by 50 HTTP connections, 950 tasks queue for a pool lease; increasing request concurrency only moves the bottleneck. Start with a dependency-specific concurrency budget: for example, 50 connections, 40 application bulkhead permits, a 500 ms pool-lease timeout, and a 1.5-second socket timeout. The pool-lease timeout is important because it distinguishes local saturation from a remote server that accepted a connection but stopped responding.

@Bean
RestClient catalogRestClient() {
  ConnectionConfig connectionConfig = ConnectionConfig.custom()
      .setConnectTimeout(Timeout.ofSeconds(1))
      .setSocketTimeout(Timeout.ofMillis(1500))
      .build();

  PoolingHttpClientConnectionManager manager =
      PoolingHttpClientConnectionManagerBuilder.create()
          .setDefaultConnectionConfig(connectionConfig)
          .build();
  manager.setMaxTotal(200);
  manager.setDefaultMaxPerRoute(50);

  CloseableHttpClient client = HttpClients.custom()
      .setConnectionManager(manager)
      .evictExpiredConnections()
      .build();

  HttpComponentsClientHttpRequestFactory factory =
      new HttpComponentsClientHttpRequestFactory(client);
  factory.setConnectionRequestTimeout(500);

  return RestClient.builder()
      .baseUrl("https://catalog.internal")
      .requestFactory(factory)
      .build();
}

Measure the change with a fixed load profile rather than comparing a single successful request. Run wrk -t8 -c200 -d60s http://localhost:8080/products/42, then inspect p50, p95, p99, error rate, and Apache HttpClient pool metrics. A useful before-after question is: did p99 improve because remote latency improved, or because failed pool acquisition now returns a fast, observable 503 instead of consuming servlet workers until the gateway times out? If virtual threads are enabled in a compatible runtime, record Java Flight Recorder and inspect pinning with jfr print --events jdk.VirtualThreadPinned recording.jfr; long pinning events often reveal blocking work inside synchronized sections or native calls.

Spring Cloud bulkheads for fan-out in a microservices architecture

A circuit breaker answers whether a dependency has recently failed; it does not cap how many requests can enter that dependency right now. Put a semaphore bulkhead in front of each independently saturating dependency. With Spring Cloud CircuitBreaker backed by Resilience4j, use a semaphore bulkhead for synchronous Spring MVC work so rejection happens on the caller thread. A thread-pool bulkhead can be appropriate for isolation, but it adds another queue and can silently break thread-local context if it is not propagated deliberately.

resilience4j.bulkhead:
  instances:
    catalog:
      maxConcurrentCalls: 40
      maxWaitDuration: 0
resilience4j.circuitbreaker:
  instances:
    catalog:
      slidingWindowSize: 50
      failureRateThreshold: 50
      waitDurationInOpenState: 10s
      recordExceptions:
        - org.springframework.web.client.ResourceAccessException

Use a zero wait duration when the request already has a strict deadline: queuing behind a bulkhead usually consumes budget without increasing the chance of a useful result. Map BulkheadFullException to HTTP 503 and include Retry-After: 1 only when the operation is safe to retry. Do not blindly retry POST requests: idempotency requires a server-side idempotency key or an operation that is naturally idempotent. Export resilience4j_bulkhead_available_concurrent_calls and compare it with dependency latency; permits near zero while remote latency rises is a capacity signal, while permits near zero with normal remote latency often indicates an undersized limit or fan-out amplification.

Spring Security context and MDC across asynchronous boundaries

Spring Security stores its context in a thread-local by default. Therefore, code submitted from a Spring MVC request to an executor can lose the authenticated principal, while MDC-based trace fields disappear at the same boundary. Wrap the executor or install a TaskDecorator; do not copy the entire HttpServletRequest into background work, because it may be invalid once the servlet request completes.

@Bean
TaskExecutor downstreamExecutor() {
  ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
  executor.setCorePoolSize(16);
  executor.setMaxPoolSize(40);
  executor.setQueueCapacity(0); // reject rather than hide overload in a queue
  executor.setTaskDecorator(task -> {
    Map<String, String> mdc = MDC.getCopyOfContextMap();
    Authentication auth = SecurityContextHolder.getContext().getAuthentication();
    return () -> {
      try {
        if (mdc != null) MDC.setContextMap(mdc); else MDC.clear();
        SecurityContext context = SecurityContextHolder.createEmptyContext();
        context.setAuthentication(auth);
        SecurityContextHolder.setContext(context);
        task.run();
      } finally {
        MDC.clear();
        SecurityContextHolder.clearContext();
      }
    };
  });
  executor.initialize();
  return executor;
}

For a spring security protected service, propagating an authenticated principal is not the same as forwarding the inbound bearer token. Downstream services should receive a token with the correct audience and least privilege, typically through OAuth 2.0 token exchange or client credentials where appropriate. Test the executor boundary explicitly: submit a task under @WithMockUser, assert the expected principal inside the task, then assert that a later task on the same worker sees no principal. That last assertion catches context leakage caused by missing cleanup in pooled threads.

Profile the spring rest api under failure, not only happy-path load

Use failure injection to validate the configuration. With Toxiproxy, add 1.2 seconds of latency or a timeout to the catalog dependency, run the same load test, and verify that bulkhead rejections occur before servlet-worker exhaustion. Capture application metrics through Actuator's /actuator/prometheus; correlate http_server_requests_seconds_bucket, HTTP client timers, bulkhead permits, JVM thread count, and gateway 5xx responses over the same time interval.

# Add latency to an existing catalog proxy for a controlled experiment
toxiproxy-cli toxic add catalog_proxy   --type latency --attribute latency=1200 --attribute jitter=100

# Inspect a 60-second load run after the toxic is enabled
wrk -t8 -c200 -d60s --latency http://localhost:8080/products/42

Set explicit acceptance criteria before changing pool sizes or timeouts: for example, p99 below the client-visible deadline, no sustained increase in active servlet requests after the dependency slows, and a bounded 503 rate rather than a wave of 504s. For CPU or allocation regressions, capture a JFR recording with jcmd <pid> JFR.start name=api settings=profile duration=60s filename=api.jfr and inspect allocation hot paths in Java Mission Control. This closes the loop between Spring Cloud resilience settings and the actual resource that is saturating.

Frequently Asked Questions

How should I set timeouts in a Spring MVC spring rest api?

Use an absolute inbound deadline, not unrelated per-hop timeouts. Set a short HTTP connection-pool lease timeout, a connect timeout, and a socket timeout that is lower than the remaining deadline. Before every downstream call, calculate remaining budget and return 504 locally when it is exhausted.

Does Spring Cloud circuit breaker prevent overload in microservices architecture?

No. A circuit breaker reacts to failures over a sampling window; it does not limit simultaneous calls. Add a Resilience4j semaphore bulkhead per dependency, set maxWaitDuration to zero for deadline-sensitive requests, and expose available-permit metrics to verify the limit under load.

How do I propagate Spring Security context to async Spring MVC tasks?

Use a TaskDecorator or DelegatingSecurityContextAsyncTaskExecutor. If logs need trace fields, copy and clear MDC as well. Always clear SecurityContextHolder in a finally block, then test that a second task on the same executor worker cannot observe the prior user's authentication.

Is spring boot training enough to use virtual threads for REST APIs?

Virtual threads reduce the cost of waiting, but they do not increase HTTP pool, database, or remote-service capacity. Pair them with bounded connection pools and bulkheads, then inspect JFR VirtualThreadPinned events and dependency metrics before increasing concurrency.