Prompt cache isolation belongs in the access-control design.

Identify which cache you are discussing

A sequence KV cache avoids recomputing earlier attention state while generating an answer. A prefix cache may retain eligible state for later requests. A response cache stores completed outputs.

They have different keys, lifetimes, and failure modes.

A cache hit can reduce computation without returning a previous answer. Exact prefix reuse is also different from a semantic cache that considers two questions similar.

A security review should name the mechanism, not just say caching is enabled.

Treat state derived from sensitive prompts as sensitive operational data. That is a design choice about handling risk; it does not claim every cache format has the same reconstruction properties or legal classification.

Understand the timing threat without overstating it

The NDSS paper I Know What You Asked demonstrates prompt-leakage attacks against shared KV-cache serving under its tested conditions. Faster cache hits can provide a signal about prior computation.

An observable timing difference does not automatically reveal arbitrary text. Exploitability depends on what an attacker can submit, the cache scope and key, access to repeated measurements, and noise from scheduling and the network.

Use this research to motivate isolation testing. It is not evidence that every hosted provider currently exposes another customer's prompts. Vendor documentation and the actual gateway configuration need separate review.

Derive cache scope at a trusted boundary

Suppose a gateway serves two law-firm matters using one provider credential. Provider-level account isolation does not, by itself, establish a boundary between those matters. The application still owns that separation.

Where supported, derive the cache namespace or salt from the authenticated tenant and authorized matter. Do not accept an arbitrary client-supplied scope as proof of authorization. Reject or replace it at the gateway.

vLLM documents cache_salt as a way to restrict prefix reuse to matching salts. Treat the deployed version and hash configuration as part of the control. A salt is a cache partition, not a replacement for document or model authorization.

If the backend cannot enforce the required scope, use separate serving pools or disable cross-request reuse for that workload. Measure the performance cost explicitly rather than weakening the boundary silently.

Test reuse within a scope and separation across scopes

Create synthetic tenants A and B. Warm a known prefix as A, then request it again as A and as B. Use server-side hit metrics where available. A should reuse eligible state; B should not reuse A's private prefix.

Repeat with cold caches, randomized order, realistic load, and distinct prefixes. Client timing alone is noisy evidence. Run a negative test in which the client attempts to supply A's scope while authenticated as B.

Test model or adapter changes, revoked access, restarts, and eviction too. A cache key that is correct for one model can be incorrect after a routing or configuration change.

Make retention specific to the mechanism

Ask whether state is memory-only, can be offloaded or persisted, has an expiry, supports explicit clearing, and can survive request completion. Record the answers for the selected provider, model, and retention mode.

Deletion may involve document stores, responses, prefixes, logs, and backups. Expiring one layer does not establish that every representation is gone. Avoid promising immediate erasure if the deployed cache only supports timed expiry.

Have the organization’s privacy owner assess personal-data and contractual duties for the actual system. An inference cache is not a trained model, so guidance about trained-model anonymity cannot simply be applied to it.

Keep the configuration with the deployment record

Record cache type, backend version, trust scope, lifetime, clearing behavior, and the isolation test result. Review this record when adding an API gateway, shared credential, new tenant, or new model.

The engineering goal is testable: authorized requests get the intended reuse, unauthorized requests do not, and operators can explain what derived state remains after the workflow ends.