Cache Me If You Can: Choosing the Right Caching Strategy
TL;DR
Done right, caching improves system performance, reduces read latency, and handles high traffic effectively. This guide focuses on choosing between Write-Through and Read-Aside caching (while acknowledging other variations with similar principles), offering practical tips on data consistency, memory constraints, and cache optimisation.
Recap of Caching Strategies
Write-Through Caching
How it works: Every time data is written to the database, it is also written to the cache, ensuring it is always up-to-date.
Key benefits:
Guarantees strong data consistency between the cache and database (until cache eviction).
The cache is always "hot" (i.e., populated with data), reducing cache misses.
Best for systems prioritising read performance, as the cache is always ready to serve data.
Trade-offs:
Higher write latency because every write operation involves both the database and the cache.
If cached items are evicted (e.g., due to TTL expiration or memory constraints) and no further writes occur, the cache will remain empty.
Solutions: Use cache warming (preloading critical data) or lazy loading (loading data on read misses) to repopulate the cache.
TTL recommendations: Use a higher TTL to reduce cache evictions and maintain a hot cache. Be mindful of memory constraints, as higher TTLs increase memory usage.
Read-Aside Caching
How it works: Data is loaded into the cache only when requested (on a cache miss). If the data isn’t in the cache, it’s fetched from the database and then added to the cache.
Key benefits:
More memory-efficient, as only frequently accessed data is cached.
Lower write latency, as writes affect only the database, not the cache.
Best for systems prioritising write performance, as it avoids overhead on every write operation.
Trade-offs:
Does not guarantee data consistency unless additional mechanisms are implemented (e.g., evicting cache keys on writes).
Cache misses lead to higher latency for the first read of uncached data. — Subsequent reads leverage this effort, though.
TTL recommendations: Use a shorter TTL to ensure frequently accessed data stays in the cache and to avoid stale data. It also helps conserve memory.
Decision Tree: Choosing the Right Caching Strategy
Walkthrough
Determine Your Priority: Is fast read performance, low write latency, strong data consistency, or memory efficiency your main concern?
Follow the Tree:
• If strong data consistency and a “hot” cache are essential, choose Write-Through. This option provides the highest data consistency but comes with increased write latency.
• If you need to prioritize low write latency or memory efficiency, go with Read-Aside. It’s more memory-efficient and allows for faster writes but requires additional mechanisms for strict consistency.
Consider Hybrid Needs: If your system requires a balance of read and write performance, explore combining strategies (e.g., Write-Through for high-priority data with lazy loading or background refreshes for less critical data).
The Last Mile: Additional Considerations
Hybrid Strategies
Write-Through with Lazy Eviction:
Use Write-Through for frequently accessed data and apply eviction rules to less-critical keys.
Example: Keep high-priority product data in Write-Through but use Read-Aside for user-specific preferences.
Read-Aside with Background Refresh:
Instead of evicting stale data, refresh critical data periodically in the background.
Example: News platforms where stories are refreshed hourly but served from the cache during peak loads.
Memory Constraints
Write-Through caching requires more memory, as higher TTLs are used to keep the cache hot. In contrast, Read-Aside caching is more memory-efficient, caching only what’s requested.
Eviction behaviour: Keep in mind; when memory is full,
Redis defaults to
noeviction(causing write failures).Memcached uses LRU to evict the least recently used keys.
Ensuring Strong Consistency
Use incoming data: When data is written to the database, also write it to the cache. This ensures strong consistency and avoids querying the database unnecessarily.
Avoid stale data: Do not rely on read replicas for cache updates, as they introduce eventual consistency and risk stale cache entries.
What to Cache
Focus on:
Frequently accessed data: Reused often across requests or users.
Expensive-to-compute data: Requires significant processing or slow database queries.
Avoid caching:
Volatile or rarely reused data (e.g., paginated content).
Prioritise shared, reusable data for maximum impact.
Cache Invalidation Techniques
Time-Based Invalidation (TTL): Expire keys after a specific duration to balance freshness with memory usage.
Event-Based Invalidation: Trigger invalidation when database changes occur (e.g., using a message queue like Kafka).
Versioning: Store versioned cache keys to ensure updates bypass stale data automatically.
Performance Metrics
Cache Hit Ratio:
High hit ratios indicate good cache utilisation.
Target 95%+ for read-heavy systems.
Latency Metrics:
Measure both cache and database query latency to understand the cache’s impact.
Eviction Rates:
Monitor how often data is evicted to fine-tune memory allocation and TTL settings.
Error Rates:
Track failed cache lookups or writes to identify bottlenecks or misconfigurations.
That’s It
Happy caching. Keep Engineering!
Appendix
Mermaid Code
Decision Tree:
---
title: Rule of Thumb for Strong Data Consistency in Caching; Use Incoming Data or DB Write-Replicas To Refresh Cache
---
flowchart TD
A([Adding <br> Cache <br>To <br>Your App?]) --> B
B{Optimize For...} --> |The Fastest Reads, ALWAYS!!!| C
C[<b>Write-Through Caching </b><br><br><div style="text-align: left"> - Highest TTL Possible/Reasonable<br> - Eager Cache Warming <br> - Pair With Cache-Aside for Evicted Cache Lazy Refresh </div><br> **Strong Data Concistency for Free**]
B --> |Highest Write Throughput| D
D[<b>Read-Aside Caching </b> <br><br> <div style="text-align: left">- Pro-Tip: Evict Keys on Writes for Stricter Consistency <br> - Lower TTL </div>]
A -.- DNote[<i>If Also Utilising DB Read-Replicas:</i> <div style="text-align: left"> <br>- Assess Redundancy With Cache<br> - Beware of Stale Data When Loading Cache</div]
style DNote fill:#f0f0f0,stroke:#333,stroke-width:1px;Read-Aside Caching:
sequenceDiagram
participant Client
participant Application
participant Cache
participant Database
Note over Cache: Read Aside
Client->>+Application: Request to read data
Application->>Cache: Read data
alt cache miss
Cache-->>Application: Cache miss
Application->>Database: Fetch data
Database-->>Application: Return data
Application->>Cache: Write data
Cache-->>Application: Acknowledge write
else cache hit
Cache-->>Application: Return data
end
Application-->>-Client: Return dataWrite-Trough Caching:
sequenceDiagram
participant Client
participant Application
participant Cache
participant Database
Note over Cache: Write Through
Client->>+Application: Request to write data
par
Application->>Database: Write Data
Database-->>Application: Acknowledge Write
and
Application->>Cache: Write Data
Cache-->>Application: Acknowledge Write
end
Application-->>-Client: Acknowledge Write