Understanding SslMode and UpstreamSslMode in ProxyMapService

In the previous article, SSL/TLS Traffic Decryption in ProxyMapService, we looked at how ProxyMapService can decrypt SSL/TLS traffic by acting as a man-in-the-middle (MITM) proxy.

However, before traffic can be decrypted, ProxyMapService must first determine an important detail:

• Is the client using TLS?
• Does the upstream server expect TLS?

In many environments the answers are obvious, but not always. ProxyMapService provides two configuration options that allow you to control this behavior explicitly:

• SslMode
• UpstreamSslMode

Both settings support the same three values:

"No"
"Yes"
"Auto"

The default value for both settings is:

"Auto"

Where These Settings Can Be Configured

Like DecryptSSL, both settings can be configured either for a listener or for an individual host rule.

Listener Configuration

"Listen": {
    "Port": 5000,
    "RejectHttpProxy": false,
    "DecryptSSL": true,
    "SslMode": "Yes",
    "UpstreamSslMode": "No"
}

When configured on a listener, the settings apply to every connection received on that listening port (or port range).

This is useful when all traffic arriving on a particular port follows the same protocol.

Host Rule Configuration

The same settings can be configured for individual destinations.

Example:

"HostRules": {
    "Items": [
        {
            "Pattern": "^mysite\\.com$",
            "HostPort": 443,
            "DecryptSSL": true,
            "SslMode": "Yes",
            "UpstreamSslMode": "No"
        }
    ]
}

This rule applies only to connections matching both:

• mysite.com
• port 443

Connections to the same host on other ports are unaffected.

Instead of using a regular expression, you can specify a host name directly:

{
    "HostName": "mysite.com",
    "DecryptSSL": true,
    "SslMode": "Yes",
    "UpstreamSslMode": "No"
}

Since no port is specified, the rule applies to every port for that host.

Host rules override listener settings whenever a connection matches the rule, allowing different SSL behavior for different destinations.

Understanding SslMode

SslMode controls whether the incoming client connection is expected to use SSL/TLS.

The available values are:

No

The incoming connection is treated as plain TCP.

No TLS handshake is expected.

Yes

The incoming connection is always treated as TLS.

ProxyMapService immediately begins TLS negotiation with the client.

Auto

This is the default behavior.

Instead of relying on the listening port, ProxyMapService examines the first two bytes received from the client.

A TLS ClientHello always begins with:

0x16 0x03

If this signature is detected, the connection is treated as TLS.

Otherwise, it is processed as an unencrypted connection.

This automatic detection allows HTTP and HTTPS traffic to coexist on the same listener without requiring separate ports.

Understanding UpstreamSslMode

UpstreamSslMode controls how ProxyMapService connects to the destination server.

Unlike SslMode, which examines the client connection, this setting determines whether the outbound connection should be encrypted.

The available values are the same:

No

ProxyMapService always connects to the upstream server using plain TCP.

Yes

ProxyMapService always establishes a TLS connection to the upstream server.

Auto

This is the default behavior.

Instead of inspecting traffic, ProxyMapService determines whether TLS should be used based on the destination port.

The following ports are treated as secure by default:

• 443 (HTTPS)
• 465 (SMTPS)
• 563 (NNTPS)
• 636 (LDAPS)
• 990 (FTPS)
• 992 (Telnet over TLS)
• 993 (IMAPS)
• 995 (POP3S)
• 3269 (Microsoft Global Catalog over SSL)
• 8443 (Alternative HTTPS)

Connections to these ports are established using TLS.

Connections to other ports use plain TCP unless UpstreamSslMode is explicitly set to Yes.

When Should You Override Auto?

For most deployments, Auto is the recommended setting for both options.

However, explicit configuration is useful when working with non-standard environments.

Typical examples include:

• HTTPS services running on custom ports.
• Plain HTTP services listening on port 443.
• Legacy applications that tunnel TLS over unexpected ports.
• Environments where protocol detection must be disabled for compatibility or performance reasons.

In these situations, setting SslMode or UpstreamSslMode to Yes or No removes any ambiguity and ensures ProxyMapService uses the desired protocol.

Conclusion

SslMode and UpstreamSslMode give you precise control over how ProxyMapService handles encrypted connections.

SslMode determines whether the client connection is encrypted, while UpstreamSslMode determines whether the connection to the destination server should use TLS.

In most cases, the default Auto mode works without any additional configuration by automatically detecting TLS on incoming connections and using well-known secure ports for outbound connections.

When working with non-standard network topologies or custom protocols, these settings allow ProxyMapService to communicate correctly with both clients and upstream servers without requiring changes to the application itself.