Traffic Caching in ProxyMapService

One of the less obvious costs of running large-scale proxy infrastructure is bandwidth. Most proxy providers charge based on traffic usage, so repeatedly downloading the same resources can quickly become expensive.

To help reduce those costs, ProxyMapService includes an HTTP response caching feature. When enabled, frequently requested resources can be served directly from the local cache instead of being downloaded through the upstream proxy again. In many scenarios this significantly reduces bandwidth consumption and lowers operating costs.

Enabling the Cache

Caching is disabled by default. To enable it, set Enabled to true in the Cache section of appsettings.json:

"Cache": {
    "Enabled": true,
    "DbPath": "c:\\temp\\cache\\proxymap.db",
    "CacheDir": "c:\\temp\\cache\\files"
}

Configuration options:

Setting	Description
Enabled	Enables or disables caching.
DbPath	Path to the SQLite database used to store cache metadata.
CacheDir	Directory where cached response bodies are stored.

What Gets Cached

ProxyMapService only caches responses that meet all of the following conditions:

• The request method is GET
• The response status code is 200 OK
• The response contains a valid content length
• The response matches a configured cache rule
• The response is considered cacheable according to the cache policy evaluation process

This approach avoids storing dynamic or non-cacheable content while keeping the cache efficient.

Cache Storage Structure

Cached responses are split into two parts:

Metadata

Metadata is stored in a SQLite table named cache_entries.

CREATE TABLE cache_entries
(
    key TEXT PRIMARY KEY,
    host TEXT,
    url TEXT,
    etag TEXT,
    cache_control TEXT,
    date TEXT,
    expires TEXT,
    last_modified TEXT,
    header_length INTEGER,
    content_length INTEGER,
    content_type TEXT,
    created_at TEXT,
    last_access TEXT
)

The table stores information required to evaluate cache validity, including HTTP cache headers such as ETag, Cache-Control, Expires, and Last-Modified.

Response Body

The actual response content is stored as a binary file under CacheDir.

Each cache entry receives a SHA-256 key generated from:

host + url

Example key:

18c97f79e87cdfdd7fd0cce3048cc6643b18ed7b154efe859d9fab09209d012a

Example cache file location:

c:\temp\cache\files\18\18c9\18c97f79e87cdfdd7fd0cce3048cc6643b18ed7b154efe859d9fab09209d012a.bin

The nested directory structure prevents large numbers of files from accumulating in a single folder.

Defining Cache Rules

Caching is controlled through CacheRules configured within a host rule.

Example:

{
    "Pattern": "gstatic\\.com$",
    "HostPort": 443,
    "Action": "Bypass",
    "DecryptSSL": true,
    "CacheRules": {
        "Items": [
            {
                "Pattern": "/images",
                "ContentTypePattern": "image/",
                "MaxAge": 60
            }
        ]
    }
}

This configuration instructs ProxyMapService to cache:

• Requests sent to hosts matching gstatic\.com$
• URLs matching /images
• Responses whose Content-Type matches image/
• Responses younger than 60 seconds

HTTPS Traffic Requires SSL/TLS Decryption

Notice the following setting:

"DecryptSSL": true

This is required when caching HTTPS traffic.

Without SSL/TLS decryption, ProxyMapService cannot inspect the request path, response headers, or content type, making cache decisions impossible.

If you are not familiar with SSL/TLS interception in ProxyMapService, see the previous article, SSL/TLS Traffic Decryption in ProxyMapService, which explains the setup process in detail.

Available Cache Rule Options

Cache rules support several filters and cache-control options.

Pattern

Matches the request URL using a regular expression.

{
    "Pattern": "/images"
}

Only requests whose URL matches the expression are considered.

AcceptPattern

Matches the request’s Accept header.

{
    "AcceptPattern": "image/"
}

This can be useful when the same URL may return different content types depending on client preferences.

ContentTypePattern

Matches the response Content-Type header.

{
    "ContentTypePattern": "image/"
}

The response must match this pattern before it can be stored or served from cache.

MaxAge

Defines a fixed lifetime in seconds.

{
    "MaxAge": 3600
}

In this example, cached content expires one hour after it is stored.

When MaxAge is specified, ProxyMapService calculates:

CreatedAt + MaxAge

If the current time exceeds that value, the cache entry is treated as expired regardless of any HTTP cache headers.

IgnoreCacheControl

{
    "IgnoreCacheControl": true
}

When enabled, ProxyMapService ignores the server’s Cache-Control, Expires, and validation directives.

Only the rule’s MaxAge setting determines whether the cached response can be used.

This can be useful when caching content from servers that provide overly restrictive cache headers.

How Cache Validation Works

When a cached entry is requested, ProxyMapService evaluates whether the entry can still be used.

The evaluation process follows standard HTTP caching semantics.

1. Cache-Control: no-store

Cache-Control: no-store

The response must never be reused.

Result:

Refresh

The content is downloaded again from the origin server.

2. Cache-Control: no-cache

Cache-Control: no-cache

The response may be stored, but it must be revalidated before use.

If the response contains validators such as ETag or Last-Modified:

Revalidate

Otherwise:

Refresh

3. Freshness Lifetime Calculation

ProxyMapService determines freshness using the following priority order.

max-age

Cache-Control: max-age=3600

This has the highest priority.

Expires

If no max-age exists:

Expires: Wed, 25 Jun 2025 12:00:00 GMT

The lifetime is calculated as:

Expires - Date

Heuristic Expiration

If neither max-age nor Expires exists, ProxyMapService falls back to a heuristic.

It calculates approximately 10% of:

Date - Last-Modified

For example, if a resource was last modified 10 days before the response date, the cache lifetime becomes roughly 1 day.

If that information is unavailable, a default lifetime of 10 minutes is used.

Stale-While-Revalidate Support

ProxyMapService also honors the stale-while-revalidate directive.

Example:

Cache-Control: max-age=60, stale-while-revalidate=300

Behavior:

• First 60 seconds → content is considered fresh
• Next 300 seconds → stale content may still be served
• Background revalidation can occur during this period

This reduces latency and unnecessary traffic while keeping content reasonably fresh.

Validation Using ETag and Last-Modified

When cached content becomes stale, ProxyMapService checks whether validation information is available.

Supported validators:

ETag: "abc123"

and

Last-Modified: Wed, 25 Jun 2025 12:00:00 GMT

If either validator exists, the entry can be revalidated instead of fully downloaded again.

This often results in a lightweight HTTP validation request rather than retransferring the entire resource.

Typical Use Cases

Traffic caching is especially effective for:

• Images
• JavaScript bundles
• CSS files
• Fonts
• Static CDN content
• Frequently requested API responses with predictable cache headers

A common pattern is to cache content from CDN providers such as Google Static Content, Cloudflare, or other asset delivery networks.

Final Thoughts

For environments where proxy traffic directly affects operating costs, caching can provide immediate savings. By storing frequently requested resources locally and honoring standard HTTP caching semantics, ProxyMapService reduces bandwidth consumption while still respecting content freshness rules.

In practice, even a small set of well-targeted cache rules for static assets can significantly reduce upstream traffic and improve response times for clients.