Rule engines, aggregators and validation pipelines share a hidden cost. Many independent components ask the same downstream question, with the same key, at the same moment. Each component is correct on its own; together they multiply the load on whatever sits behind them.
This article shows how to collapse those calls into a single load using Spring's cache abstraction backed by Caffeine. It also covers what tends to bite afterwards: cached empty results, metrics that change meaning, and tests that pass for the wrong reason.
The scenario
Take a promotion eligibility service. To decide whether a product qualifies for a promotion, it evaluates about 20 rules in parallel: in stock, price above a threshold, seller in good standing, category not excluded, and so on.
List<CompletableFuture<Boolean>> results = rules.stream()
.map(rule -> CompletableFuture.supplyAsync(() -> rule.test(productId), executor))
.toList();
Each rule depends on a small, focused service:
@Component
class InStockRule implements Rule {
private final StockService stockService;
@Override
public boolean test(String productId) {
return stockService.isInStock(productId);
}
}
@Service
class StockService {
private final ProductCatalogue catalogue;
boolean isInStock(String productId) {
return catalogue.findProduct(productId)
.map(product -> product.stock() > 0)
.orElse(false);
}
}
PricingService, SellerService and CategoryService look the same: each one reads a different field from the same Product, which comes from one place:
@Service
public class ProductCatalogue {
// Reads from a shared cache (Redis, DynamoDB, ...), falls back to the catalogue API on a miss.
public Optional<Product> findProduct(String productId) {
return store.get(productId).or(() -> fetchAndStore(productId));
}
}
The design is clean: every rule is small, testable and independent. But one evaluation of one product calls findProduct about 20 times:
- Warm path: 20 reads from the shared cache, plus 20 deserialisations of the same object.
- Cold path: all 20 calls miss at the same time, so the catalogue API gets 20 concurrent requests for the same product. That's a cache stampede, and it hits hardest when the downstream system is already struggling.
The calls don't need 20 answers. They need one.
Why not request scope?
The first idea is usually a request-scoped cache: memoise the response for the lifetime of the HTTP request. That works less well than it sounds:
-
Thread hopping. The rules run on executor threads. Request attributes are bound to the request thread, so request-scoped beans only work if you propagate the context to every executor (for example with a
TaskDecorator). -
No request at all. Queue consumers, Kafka listeners and scheduled jobs have no HTTP request. A
@RequestScopebean there fails withNo thread-bound request found.
A short-lived, in-process cache gives the same effect on every entry point. The price is bounded staleness, and with a short enough TTL that price is negligible.
Setup
Dependencies
spring-boot-starter-cache brings Spring's cache abstraction, but not Caffeine. Add both. Spring Boot manages the Caffeine version.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
</dependency>
implementation("org.springframework.boot:spring-boot-starter-cache")
implementation("com.github.ben-manes.caffeine:caffeine")
Enable caching
@SpringBootApplication
@EnableCaching
public class Application { }
Configure the cache
spring:
cache:
type: caffeine
cache-names: products
caffeine:
spec: maximumSize=10000,expireAfterWrite=30s,recordStats
Each line matters:
-
type: caffeinemakes the application fail at startup if Caffeine is missing. Without it, Spring Boot silently falls back toConcurrentMapCacheManager. That cache ignorescaffeine.spec, so it has no size limit and entries never expire: unbounded memory and unbounded staleness, a worse problem than the one you set out to fix. Dependencies move around, so don't rely on Caffeine always being on the classpath. -
cache-namescreates the listed caches at startup and turns off dynamic creation: a@Cacheablethat uses an unlisted name fails at runtime. It's also what lets Spring Boot register Micrometer metrics for the cache, because only caches that exist at startup are bound. -
spec:maximumSizecaps memory,expireAfterWritecaps staleness, andrecordStatsturns on the hit/miss statistics behind the metrics.
The annotation
@Cacheable(cacheNames = "products", key = "#productId", sync = true)
public Optional<Product> findProduct(String productId) {
return store.get(productId).or(() -> fetchAndStore(productId));
}
Two attributes do most of the work here.
sync = true is the difference between caching and collapsing. Without it, Spring checks the cache, misses, calls the method and stores the result. When 20 threads miss at the same moment, all 20 call the method. With sync = true, Spring hands the load to Caffeine's atomic get(key, loader): the first thread runs the method, and the other 19 wait for its result.
key = "#productId" makes the key explicit. The default key generator uses the bare argument for single-argument methods, but the moment someone adds a second parameter, the key changes shape and any matching @CacheEvict silently stops working. Referring to parameters by name needs the -parameters compiler flag (Spring Boot's build plugins enable it); otherwise use #p0.
Evicting works the same way:
@CacheEvict(cacheNames = "products", key = "#productId")
public void evict(String productId) { }
@CacheEvict(cacheNames = "products", allEntries = true)
public void evictAll() { }
A classic mistake: @CacheEvict("products") on a method with no arguments evicts the key SimpleKey.EMPTY, which clears nothing. Use allEntries = true to clear the whole cache.
What will surprise you
1. Empty results are cached too
Spring unwraps Optional: present values are stored as the value, and empty ones are stored as null. CaffeineCacheManager allows null values by default, so "product not found" is memoised for the full TTL.
This is often what you want: without it, a missing product is looked up again by every one of the 20 callers. But decide it deliberately. The usual escape hatch, unless = "#result == null", isn't allowed together with sync = true. If you can't accept caching empty results, you'll need a different shape, for example throwing a not-found exception (exceptions are never cached) or using the Cache API directly.
2. Failures are not shared
Exceptions aren't cached. That's good: a transient failure doesn't stick for the next 30 seconds. But it also means that when the load fails, the waiting callers don't get the error. They retry the load one after another. Under a sustained outage, put a circuit breaker or a stale-fallback strategy inside the loaded method. Don't count on the cache to absorb the failures.
3. Every caller gets the same instance
Before the cache, each caller got its own freshly deserialised copy. Now all callers within the TTL share one object, across threads. If anything modifies it, you have a data race between components that don't know about each other. Prefer immutable types (records, or immutable generated classes). If you can't, say so in a comment on the cached method.
4. Self-invocation bypasses the cache
@Cacheable works through a Spring proxy. A call from inside the same class (this.findProduct(...)) skips the proxy and the cache. Check that every hot caller goes through the bean.
5. The cache is per instance
Each pod or instance has its own cache. An eviction endpoint only clears the instance that received the request; the others keep serving the old value until the TTL expires. That's acceptable when the TTL is short. Write it down for whoever operates the endpoint.
6. Your existing metrics change meaning
Any counter incremented inside the cached method (misses on the shared cache, remote calls, fallbacks) now only fires on in-memory misses. On the day you deploy, dashboards show a sudden drop that looks like an outage or a miracle. It's neither. Warn people before the deploy, and add the cache's own metrics, which recordStats exposes:
cache.gets{cache="products", result="hit"|"miss"}cache.putscache.evictionscache.size
7. Pick the TTL for the unit of work
The goal is to collapse the calls within one unit of work, not to build a long-lived performance cache. So the TTL needs to be long enough to cover one evaluation including slow paths (retries and their backoff), and short enough that the extra staleness is negligible compared with how fresh the data behind it is anyway. Seconds to tens of seconds is usually right. If you want a longer TTL, that's a different cache, and it needs its own invalidation design.
8. Virtual threads on JDK 21–23
With sync = true, the load runs inside Caffeine's atomic compute, which holds a synchronized lock. On JDK 21–23, a virtual thread that blocks on I/O inside synchronized pins its carrier thread. Since carrier threads are few (one per core by default), many concurrent cold loads can starve the scheduler. JDK 24 fixed this (JEP 491). On older JDKs, load-test the cold path with -Djdk.tracePinnedThreads=full.
Testing it
Prove that concurrent misses collapse
A naive test fires 20 calls and checks that the loader ran once. It usually passes even without sync = true: the stubbed load returns instantly, so the calls run one after another and the first one fills the cache before the second arrives. The test has to keep callers inside the loader until all of them have arrived:
@SpringJUnitConfig(ProductCatalogueCachingTest.Config.class)
class ProductCatalogueCachingTest {
private static final int CALLERS = 20;
@Autowired
ProductCatalogue catalogue;
@MockitoBean
ProductStore store;
@Test
void concurrentMissesLoadOnce() throws Exception {
var allCallersLoading = new CountDownLatch(CALLERS);
var release = new CountDownLatch(1);
// Every caller that reaches the loader waits here, so without sync = true all of them would load.
when(store.get("p-1")).thenAnswer(invocation -> {
allCallersLoading.countDown();
release.await(5, TimeUnit.SECONDS);
return Optional.of(new Product("p-1", 3));
});
try (var executor = Executors.newFixedThreadPool(CALLERS)) {
var futures = IntStream.range(0, CALLERS)
.mapToObj(i -> executor.submit(() -> catalogue.findProduct("p-1")))
.toList();
allCallersLoading.await(500, TimeUnit.MILLISECONDS);
release.countDown();
for (var future : futures) {
assertThat(future.get(5, TimeUnit.SECONDS)).isPresent();
}
}
verify(store, times(1)).get("p-1");
}
@Configuration
@EnableCaching
static class Config {
@Bean
CacheManager cacheManager() {
return new CaffeineCacheManager("products");
}
@Bean
ProductCatalogue catalogue(ProductStore store) {
return new ProductCatalogue(store);
}
}
}
With sync = true, the first caller enters the loader and the other 19 wait in Caffeine, so get runs once. Without it, all 20 enter the loader and the test fails with Wanted 1 time, but was 20 times. Remove sync = true once to see that failure: a test you've never seen fail hasn't proven anything.
Prove that the shipped configuration is the one you think
The test above builds its own CaffeineCacheManager, so it never touches application.yml. A typo in cache-names, or a missing type, would only show up in production. ApplicationContextRunner can load the real configuration file with only the cache auto-configuration:
@Test
void applicationConfigurationCreatesBoundedCaffeineCache() {
new ApplicationContextRunner()
.withInitializer(new ConfigDataApplicationContextInitializer())
.withConfiguration(AutoConfigurations.of(CacheAutoConfiguration.class))
.withUserConfiguration(CachingEnabled.class)
.run(context -> {
var manager = context.getBean(CacheManager.class);
assertThat(manager).isInstanceOf(CaffeineCacheManager.class);
var cache = (com.github.benmanes.caffeine.cache.Cache<?, ?>)
manager.getCache("products").getNativeCache();
assertThat(cache.policy().eviction().orElseThrow().getMaximum()).isEqualTo(10_000);
assertThat(cache.policy().expireAfterWrite().orElseThrow().getExpiresAfter())
.isEqualTo(Duration.ofSeconds(30));
});
}
@Configuration
@EnableCaching
static class CachingEnabled { }
CacheAutoConfiguration lives in org.springframework.boot.cache.autoconfigure in Spring Boot 4, and in org.springframework.boot.autoconfigure.cache in Spring Boot 3.
Also add one test each for eviction (evict, call again, check the loader ran twice) and for whichever empty-result behaviour you chose.
Checklist
- [ ]
spring-boot-starter-cacheandcaffeinedeclared,@EnableCachingon - [ ]
spring.cache.type: caffeineso a missing Caffeine fails at startup - [ ]
cache-namesdeclared (and new names added there) - [ ]
maximumSize,expireAfterWriteandrecordStatsin the spec - [ ]
@Cacheable(sync = true)with an explicitkey - [ ]
@CacheEvictuses the same key; clear-all usesallEntries = true - [ ] Empty-result caching decided and tested
- [ ] Cached return type immutable, or documented as shared
- [ ] No self-invocation on hot paths
- [ ] Dashboard owners warned about counters inside the cached method
- [ ] Concurrency test that fails without
sync = true - [ ] Configuration test against the real
application.yml - [ ] On JDK 21–23 with virtual threads: cold path load-tested for pinning
Wrapping up
The change itself is a dependency, a few lines of YAML and one annotation. Most of the value is in the rest: deciding what gets cached (including "not found"), making the configuration fail loudly instead of degrading silently, warning the people who read the metrics, and writing tests that actually fail when the behaviour breaks. Get those right, and twenty components asking the same question cost the same as one.
