Building a soft reference cache for memory-sensitive Java systems

Java applications cache recently used objects to reduce database traffic and latency. A soft reference cache lets the garbage collector clear entries under heap pressure while keeping them available during ordinary activity. It suits reconstructible values, not authoritative records or data the application must never lose.

Wrapping values in SoftReference is only the start. Keys, stale mappings and cleanup need coordination across request threads. Cache lookup should also remain separate from slow I/O, or a latency spike can become a traffic surge.

These concerns are familiar in Australia. Banks, retailers and logistics operators run services from Sydney, Melbourne and Brisbane, often on fixed-memory cloud instances. Commuters moving between CBD offices and suburban homes create sharp mobile-traffic peaks, making predictable memory use valuable when catalogue data and route calculations arrive quickly.

Under the Privacy Act 1988, covered organisations handling personal information must follow the Australian Privacy Principles. Cached customer records need a clear purpose, limited retention and appropriate safeguards. The examples focus on disposable derived data such as exchange rates, product snippets and public catalogue pages.

Choosing values the JVM may reclaim

SoftReference lets the collector reclaim a referent when memory is tight. Timing is not an eviction contract: JVMs may clear references differently, and batches can disappear together. Treat every hit as provisional because nothing guarantees how long the value remains cached.

Use it for costly repeatable calculations, short-lived remote responses and large objects that can be fetched again. Keep payment authorisation, session state and sole business records elsewhere. Keys should be small and independent of the value, or a strong key can retain a larger object graph.

A concurrent cache with reference cleanup

The implementation below uses ConcurrentHashMap for thread-safe lookup and a ReferenceQueue to report cleared referents. Each queued entry carries its key, allowing stale mappings to be removed without deleting a newer value installed under the same key.

import java.lang.ref.ReferenceQueue;
import java.lang.ref.SoftReference;
import java.util.Objects;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;

final class SoftCache<K, V> {
    private final ConcurrentMap<K, CacheEntry<K, V>> entries =
            new ConcurrentHashMap<>();
    private final ReferenceQueue<V> queue = new ReferenceQueue<>();

    V get(K key) {
        CacheEntry<K, V> entry = entries.get(key);
        if (entry == null) {
            drainQueue();
            return null;
        }

        V value = entry.get();
        if (value == null) {
            entries.remove(key, entry);
            drainQueue();
        }
        return value;
    }

    V put(K key, V value) {
        Objects.requireNonNull(key);
        Objects.requireNonNull(value);

        CacheEntry<K, V> entry = new CacheEntry<>(key, value, queue);
        CacheEntry<K, V> previous = entries.put(key, entry);
        drainQueue();
        return previous == null ? null : previous.get();
    }

    void invalidate(K key) {
        entries.remove(key);
    }

    int size() {
        return entries.size();
    }

    @SuppressWarnings("unchecked")
    void drainQueue() {
        CacheEntry<K, V> entry;
        while ((entry = (CacheEntry<K, V>) queue.poll()) != null) {
            entries.remove(entry.key, entry);
        }
    }

    private static final class CacheEntry<K, V> extends SoftReference<V> {
        private final K key;

        CacheEntry(K key, V value, ReferenceQueue<V> queue) {
            super(value, queue);
            this.key = key;
        }
    }
}

Conditional removal protects replacements: if one thread installs a new entry while another discovers that the old referent has vanished, remove(key, oldEntry) leaves the new mapping intact. A value can still be cleared immediately after get returns it, which is normal and becomes a miss on the next request.

Eviction, consistency and failure modes

The queue is a cleanup signal, not an LRU policy. Nothing promises which key is evicted first, and the cache has no expiry, maximum size or refresh schedule. It reduces retained-memory pressure but offers no strict capacity control.

Drain the queue during writes and misses, and schedule maintenance for idle applications so cleared entries do not leave strong keys behind. Treat size as approximate because collection and replacement run concurrently. Cached instances remain mutable, so callers must coordinate changes.

Caffeine often provides predictable capacity, expiry and hit statistics. Soft references remain useful in legacy systems, specialised JVMs and components that intentionally want the collector to force a reload under pressure.

Wiring the cache into Spring

Expose the cache as a singleton bean and inject it into the service that needs it. The custom Spring configuration guide shows how application infrastructure can be wired, while constructor injection keeps creation independent of request threads.

Keep repository calls outside map operations. A service can perform get, load a value and call put; concurrent misses may duplicate work, so request coalescing helps when the endpoint is expensive. Never expose the internal map or mutate cached instances in place.

Other retention sources matter too. Static fields, unbounded collections, open HTTP response bodies and large thread-local values can consume more heap than the cache. Log cache size with heap and garbage-collection metrics to reveal real pressure.

Testing behaviour in Australian deployments

Test with production-like object sizes and a sudden allocation burst. Record hit ratio, cleared-reference count, heap use, collection time and request latency at the 95th and 99th percentiles. Java Flight Recorder and Micrometer expose these signals without changing application logic.

Include slow mobile connections and office Wi-Fi handovers, common during Australian commutes, and verify that a cleared entry causes one controlled reload rather than a retry storm. Keep authentication tokens and regulated customer details outside this cache.

For an Australian retailer combining a Java pricing service with a public storefront, backend caching and frontend delivery remain separate concerns. A web development resource can support the storefront side, while the cache protects the service's memory budget. Here, the reload path, thread-safety rules and privacy boundaries remain explicit rather than being delegated to memory pressure.