Neural Mastery
← Practice
Exact Match Semantic Prompt Cachemedium

Exact Match Semantic Prompt Cache

medium Agents in Production 20–25 min

NumPy allowed · plain Python earns 10 bonus points

LLM step 1/4

Logits become a next-token distribution

Softmax turns a model score vector into probabilities.
Open lesson →
01prompt
02tokens
03logits
04probabilities
attention→next token
mechanism65%
layer20%
pattern10%
model5%

Softmax(logits) → probability distribution

Reduce latency and API expenditure by caching recurring LLM prompt responses with LRU eviction and TTL invalidation.

Task

Implement simulate_prompt_cache(operations, capacity=100, default_ttl_sec=3600). Process operations in order. Normalize prompts by stripping leading/trailing whitespace, collapsing internal whitespace, and lowercasing. For 'get': check if prompt exists in cache; if expired (timestamp > expire_timestamp), delete entry, increment expired and misses; if valid, refresh LRU access and increment hits; if not found, increment misses. For 'set': store response with expire_timestamp = timestamp + ttl. If key exists, overwrite and refresh LRU; if inserting causes size > capacity, evict the least recently used item (evictions += 1). Return {'hits': int, 'misses': int, 'evictions': int, 'expired': int, 'final_keys': list[str]}. Raise ValueError if capacity <= 0 or default_ttl_sec <= 0.

Function Signature

simulate_prompt_cache(operations: list[dict], capacity: int = 100, default_ttl_sec: int = 3600) -> dict

Examples

Example 1: Set then Get cache hit with normalization
Input: operations = [{'op': 'set', 'prompt': ' What is Python? ', 'response': 'A language', 'timestamp': 100}, {'op': 'get', 'prompt': 'what is python?', 'timestamp': 110}] capacity = 2 default_ttl_sec = 300
Output: {'hits': 1, 'misses': 0, 'evictions': 0, 'expired': 0, 'final_keys': ['what is python?']}
Example 2: Cache expiration on TTL
Input: operations = [{'op': 'set', 'prompt': 'hello', 'response': 'hi', 'timestamp': 100, 'ttl_sec': 50}, {'op': 'get', 'prompt': 'hello', 'timestamp': 160}] capacity = 2 default_ttl_sec = 100
Output: {'hits': 0, 'misses': 1, 'evictions': 0, 'expired': 1, 'final_keys': []}
Example 3: LRU capacity eviction
Input: operations = [{'op': 'set', 'prompt': 'k1', 'response': 'v1', 'timestamp': 10}, {'op': 'set', 'prompt': 'k2', 'response': 'v2', 'timestamp': 20}, {'op': 'set', 'prompt': 'k3', 'response': 'v3', 'timestamp': 30}] capacity = 2 default_ttl_sec = 1000
Output: {'hits': 0, 'misses': 0, 'evictions': 1, 'expired': 0, 'final_keys': ['k2', 'k3']}
Example 4: Invalid capacity raises ValueError
Input: operations = [] capacity = 0
Output: Raises ValueError

Constraints

  • Normalize prompt whitespace and case.
  • Evict least recently used key when exceeding capacity.
  • Raise ValueError on non-positive capacity or TTL.
Python3Saved ✓
def simulate_prompt_cache(operations: list[dict], capacity: int = 100, default_ttl_sec: int = 3600) -> dict:
"""Simulate an LRU prompt response cache with TTL expiration and prompt normalization.
Args:
operations: List of op dicts ('get' or 'set') with timestamp, prompt, and optional response.
capacity: Maximum entries held before evicting least-recently-used item.
default_ttl_sec: Default time-to-live expiration in seconds.
Returns:
Dict with 'hits', 'misses', 'evictions', 'expired', and 'final_keys'.
"""
# Your implementation here
pass

Case 1: Set then Get cache hit with normalization
Input: operations = [{'op': 'set', 'prompt': ' What is Python? ', 'response': 'A language', 'timestamp': 100}, {'op': 'get', 'prompt': 'what is python?', 'timestamp': 110}] capacity = 2 default_ttl_sec = 300
Expected: {'hits': 1, 'misses': 0, 'evictions': 0, 'expired': 0, 'final_keys': ['what is python?']}
Case 2: Cache expiration on TTL
Input: operations = [{'op': 'set', 'prompt': 'hello', 'response': 'hi', 'timestamp': 100, 'ttl_sec': 50}, {'op': 'get', 'prompt': 'hello', 'timestamp': 160}] capacity = 2 default_ttl_sec = 100
Expected: {'hits': 0, 'misses': 1, 'evictions': 0, 'expired': 1, 'final_keys': []}
Case 3: LRU capacity eviction
Input: operations = [{'op': 'set', 'prompt': 'k1', 'response': 'v1', 'timestamp': 10}, {'op': 'set', 'prompt': 'k2', 'response': 'v2', 'timestamp': 20}, {'op': 'set', 'prompt': 'k3', 'response': 'v3', 'timestamp': 30}] capacity = 2 default_ttl_sec = 1000
Expected: {'hits': 0, 'misses': 0, 'evictions': 1, 'expired': 0, 'final_keys': ['k2', 'k3']}
Case 4: Invalid capacity raises ValueError
Input: operations = [] capacity = 0
Expected: Raises ValueError