Design Patterns & Practices  

Why High-Frequency Lookups Become a Bottleneck

Many applications repeatedly retrieve:

  • User permissions

  • Product catalog metadata

  • Exchange rates

  • Feature flags

  • Ledger codes

  • Workflow rules

  • Inventory availability

  • Microservice endpoint configs

When the same data is queried thousands of times per minute, the impact becomes clear:

1. Database CPU Rises Quickly

Repeated reads on small lookup tables often account for 40–60 percent of CPU usage in OLTP systems.

2. Network Round-Trips Add Latency

Even a fast query becomes costly when repeated millions of times.

3. Locks And Latches Slow Everything

Repeated selects can still escalate into contention on hot pages.

4. Downstream Microservices Become Slow

Distributed lookups cause cascading latency.

Cache-aware design attempts to solve these problems at the root, not by tweaking indexes or optimizing slow queries individually.

What Is Cache-Aware Design?

Cache-aware design means:

  • You know which data is accessed most often

  • You categorize it by volatility, size, and access pattern

  • You define caching zones (L1, L2, distributed, client-side)

  • You choose the most efficient invalidation strategy

  • You eliminate redundant calls through shared lookup contexts

It is not simply “put Redis in front of SQL Server.”
It is a holistic design approach.

Types Of Lookups That Benefit Most From Caching

The following lookup patterns benefit the most:

1. High-Frequency, Low-Volatility Data

Examples:
Role permissions, state master, currency metadata, country list.

2. Read-Mostly Entities

Examples:
Product details, pricing slabs updated once a day, configuration settings.

3. Pre-Computation Check Values

Examples:
Pre-approved promo codes, ledger account validity, routing rules.

4. Complex Computation Results

If the computation is deterministic and repeated, cache it.

The Four Layers Of a Cache-Aware Architecture

Here is a typical multi-layer caching design:

+---------------------------+
|        Client Cache       | (Browser, Mobile, Angular services) - L0
+---------------------------+
              |
              v
+---------------------------+
|     Application Cache     | (MemoryCache, IMemoryCache) - L1
+---------------------------+
              |
              v
+---------------------------+
|    Distributed Cache      | (Redis, Memcached) - L2
+---------------------------+
              |
              v
+---------------------------+
|        SQL Database       | (Authoritative Store)
+---------------------------+

L0: Client-Side Caching (Angular Focus)

Angular helps with high-frequency lookups through:

  • HttpInterceptor response caching

  • BehaviorSubject data stores

  • LocalStorage / SessionStorage for static master lists

  • State management using NgRx for shared application-wide lookups

Example Angular service for caching lookups:

@Injectable({ providedIn: 'root' })
export class LookupService {
  private cache = new Map<string, any>();

  constructor(private http: HttpClient) {}

  getCurrencyCodes(): Observable<any> {
    const key = 'currencyCodes';

    if (this.cache.has(key)) {
      return of(this.cache.get(key));
    }

    return this.http.get('/api/lookups/currencyCodes').pipe(
      tap(result => this.cache.set(key, result))
    );
  }
}

This prevents repeated network calls within the same session.

L1: In-Memory Cache (ASP.NET Core)

In-process memory caching is extremely fast and ideal for:

  • Permission checks

  • Master data

  • Feature flags

  • Lightweight rule configurations

Example:

public async Task<List<Country>> GetCountriesAsync()
{
    return await _cache.GetOrCreateAsync("countries", async entry =>
    {
        entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(6);
        return await _db.Countries.AsNoTracking().ToListAsync();
    });
}

L2: Distributed Cache (Redis)

Distributed cache is designed for:

  • Clusters (load-balanced API servers)

  • High throughput

  • Consistent lookups across nodes

Redis is ideal when you need:

  • Data consistency across multiple app servers

  • Shared cached values between microservices

  • Global rate limiting or throttling tokens

  • Large objects or lists that exceed process memory

  • Cross-region caching via Redis Enterprise

Designing Optimal Cache Keys

A cache-aware system uses human-friendly, pattern-based keys:

lookup:country:all
lookup:country:IN
lookup:roles:admin
lookup:ledger:code:4001
config:paymentgateway:stripe
pricing:slab:gold

Principles:

  • Always prefix with domain (lookup, config, pricing)

  • Use clear hierarchies

  • Avoid storing too large objects under one key

  • Use versioning when schema changes

Example versioned key:

lookup:country:v2:all

Query Flow When Cache Is Enabled

Request -> Check L1 Memory Cache -> If miss -> Check Redis (L2)
     -> If miss -> Query SQL -> Update Redis -> Update L1 -> Return Result

Expanded:

+------------------+
|   API Request    |
+------------------+
          |
          v
+-----------------------------+
| L1 In-Memory Cache (Fastest)|
+-----------------------------+
          |
      MISS|
          v
+-----------------------------+
|   Redis Distributed Cache   |
+-----------------------------+
          |
      MISS|
          v
+-----------------------------+
|       SQL Database          |
+-----------------------------+

Detecting When Your System Needs Caching

Check for these indicators:

1. Excessive Reads On Small Reference Tables

If a 100-row master table is queried 10,000 times per minute, caching is mandatory.

2. CPU Saturation On Simple Lookup Queries

Query complexity does not matter; volume overwhelms SQL CPU.

3. Duplicate Queries Across Application Layers

E.g., repeated permission checks across many API endpoints.

4. High Network Latency

If calls travel across regions, caching gives immediate performance gains.

Cache Invalidation Strategies That Actually Work

Cache invalidation is the hardest part. These strategies are reliable:

1. Time-To-Live (TTL)

Safe and simple when data changes infrequently.

2. Event-Based Invalidation

Use SQL dependency or triggers to publish updates to Redis channels.

Example:

UPDATE Country SET Name = @NewName WHERE Code = 'IN';
PUBLISH redis:invalidate 'lookup:country:IN';

3. Versioned Keys

When the structure changes, bump a version.

4. Soft Refresh

If TTL expires, return stale data and refresh asynchronously.

High-Frequency Lookup Optimization Patterns

Pattern 1: Lookup Pre-Loading On App Startup

Preload lookup sets into memory when application starts.

Pattern 2: Lookup Context Flow (Per-Request)

Collect all required lookups once and share across services, instead of querying repeatedly.

Pattern 3: Cache With Lazy Refresh

Caches refresh only when needed, not on every expiry.

Pattern 4: Cache Stampede Prevention

Use distributed locking so that only one server refreshes an expired key.

Example: Optimizing Ledger Code Validations

Scenario:

A financial system checks ledger codes for every posting.
Each posting results in:

  • Validating ledger code

  • Validating account type

  • Validating currency restrictions

Instead of three database calls, use a composite Redis key:

ledger:validation:set

Store:

  • Valid ledger codes

  • Allowed account types

  • Constraints

Then one in-memory call handles all checks.

Example: Optimizing E-Commerce Pricing Lookups

A pricing service requires:

  • Discount slabs

  • Product attributes

  • Location-based tax rules

  • Customer tier benefits

All can be cached as:

pricing:v1:product:{productId}
pricing:v1:tax:{stateCode}
pricing:v1:tier:{tierId}

TTL: 30 minutes
Version changes when rules change.

Cache Behavior Under Load (Illustration)

High Throughput Scenario:
100,000 requests/min

Without Cache:
100,000 DB reads
=> High CPU
=> High network latency
=> Contention on hot tables

With Cache:
L1 Cache Hits: 80,000
L2 Redis Hits: 15,000
DB Reads: 5,000
=> Databases stay cold
=> Application runs fast
=> No contention

Common Mistakes In Cache Design

  • Storing too large objects

  • Using one giant master object instead of multiple small keys

  • Not tracking cache hit ratio

  • Poor invalidation strategy leading to stale data

  • Missing cache isolation between tenants

  • Trying to cache every query

  • Using Redis as a persistence layer (it is not)

How To Measure Cache Effectiveness

Track:

  • L1 hit ratio

  • L2 (Redis) hit ratio

  • DB query reduction percentage

  • Latency reduction

  • CPU savings on SQL Server

  • Redis memory footprint

  • Cache churn (how often keys are rebuilt)

Reference Architecture For Cache-Aware Systems

+--------------------------------------------------------+
|                 Global Discovery Layer                 |
|     (Feature Flags, Routing, Rate Limits, Config)      |
+----------------------------+---------------------------+
                             |
                             v
+--------------------------------------------------------+
|              Distributed Cache (Redis Cluster)         |
+--------------------------------------------------------+
                |                           |
                v                           v
+-------------------------------+   +-------------------------------+
|    Application Server API 1   |   |    Application Server API 2   |
|  L1 Memory Cache + Lookup ctx |   |  L1 Memory Cache + Lookup ctx |
+-------------------------------+   +-------------------------------+
                |                           |
                -----------SQL Database------

When To Skip Caching

Caching is not useful when:

  • Data changes every few seconds

  • Lookup values must be 100 percent real-time and consistent

  • Values depend on user-specific dynamic filters

  • Data size is too large for cache (e.g., full inventory dumps)

For these, design alternatives include:

  • Stored procedures with optimized indexes

  • Materialized views

  • Snapshot tables

  • Columnstore indexes

Final Recommendations

Cache-aware design is not optional for modern, high-scale systems.
It is essential for:

  • Cutting repeated database reads

  • Reducing latency

  • Increasing throughput

  • Making microservices predictable under load

A good caching strategy uses:

  • L0 client-side caching (Angular)

  • L1 in-memory caching

  • L2 distributed caching (Redis)

  • Clear key naming conventions

  • Proper TTL and invalidation

  • Versioning to avoid stale structures

Once implemented, high-frequency lookup performance improves 10–100x.