How to Inspect TLS Encrypted Traffic

Do you want to analyze decrypted TLS traffic in Wireshark or let an IDS, like Suricata, Snort or Zeek, inspect the application layer data of potentially malicious TLS encrypted traffic? There are many different TLS inspection solutions to choose from, but not all of them might be suitable for the specific challenge you’re facing. In this blog post I describe three different methods for decrypting TLS and explain when to use one or the other.

RSA Private Key

TLS decryption with a private RSA key was for a long time the preferred method for inspecting SSL and TLS traffic. This approach allowed anyone with access to the server’s private RSA key to decrypt the traffic and inspect the application layer (L7) communication.

The primary drawback with RSA private key decryption is that a stolen or leaked private RSA key can be used by an attacker to decrypt all previously captured traffic from that server, if RSA key exchange is being used. Modern TLS stacks have therefore deprecated such ciphers in favor of ones that support forward secrecy, which typically perform an ephemeral Diffie–Hellman (DHE) key exchange instead of reusing the same private RSA key over and over. This means that the RSA private key decryption method cannot be used if the client and server are using a key exchange algorithm that supports forward secrecy.

RSA private key decryption can only be performed when all these conditions are met:

• The protocol version is SSL 3.0, TLS 1.0, TLS 1.1 or TLS 1.2 (RSA was removed in TLS 1.3)
• The server has selected a cipher suite that use RSA key exchange, such as TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA or TLS_RSA_WITH_AES_128_CBC_SHA
• The private key matches the server certificate (traffic cannot be decrypted with a client certificate or an intermediate or root certificate)
• The session has not been resumed (the handshake must include a Client Key Exchange message)

This Wireshark display filter can be used to check if the server has selected an RSA cipher:

You can check for a client key exchange message with:

A private RSA key can be loaded into Wireshark by clicking Edit, Preferences and RSA Keys. Another alternative is to use the command line tool tshark’s -ouat:rsa_keys switch like this:

TLS Key Log

Wireshark can decrypt the TLS layer in captured network traffic if the pre-master secrets used to establish the encrypted connection are provided. These secrets, or encryption key material, can be loaded into Wireshark from an SSLKEYLOGFILE by clicking Edit, Preferences, Protocols, TLS, and setting the (Pre)-Master-Secret log filename to the path of your SSLKEYLOGFILE.

Another alternative is to encode the key material as metadata in a pcap-ng file with editcap like this:

The primary drawback with the TLS key log decryption method is that only Wireshark and tshark can be used to analyze the decrypted TLS traffic. You also need to get hold of the keys or pre-master secrets in order to perform the decryption. Some applications, such as Firefox, Chrome and curl, can be configured to export a key log. Another alternative is to install an agent that extracts key material from specific TLS libraries.

The limitation of only being able to extract keys from a specific set of applications or TLS libraries makes the TLS key log method unsuitable for analyzing TLS encrypted C2 traffic from malware, which often use custom TLS libraries. It is also difficult to send decrypted TLS traffic to an IDS or a network security monitoring tool using a TLS key log. If you, on the other hand, want to analyze network traffic from your own Firefox or Chrome browser in Wireshark, then the TLS key log approach is probably the best solution.

TLS Inspection Proxy

A TLS inspection proxy acts as a man-in-the-middle that intercepts and decrypts TLS traffic for inspection, it then re-encrypts the traffic and forwards it to the intended destination.

A major advantage of using a TLS inspection proxy is that decrypted TLS traffic can be analyzed from applications even if they use modern ciphers with forward secrecy and don’t support logging of TLS keys. The drawback, however, is that clients have to trust the root CA certificate that the proxy is using.

TLS inspection proxies often differ in how they make the decrypted traffic available to external tools, if at all. In fact, many TLS inspection proxies and Next-Generation Firewalls (NGFW) only make the decrypted payload available to the internal application or appliance. Such an approach prevents analysis of the decrypted traffic with an external tool, like Wireshark, Snort, Suricata, Zeek or NetworkMiner.

Another approach, used by proxies like mitmproxy, is to save a TLS key log for all proxied traffic. That approach allows captured TLS traffic to or from the proxy to be decrypted and inspected with Wireshark, but the application layer traffic cannot be inspected with other tools that don’t support TLS decryption using a key log.

The third and most integration friendly approach is to save the decrypted traffic in clear text form, so that other applications can inspect the unencrypted traffic without having to decrypt TLS. Some TLS proxies, like PolarProxy and SSLsplit, can even save the decrypted traffic to a PCAP file. Decrypted TLS traffic in PCAP format can easily be ingested into other tools or be replayed to a network interface for inspection by an external security appliance.

Best Practices

The list below can be used to select the best suited TLS inspection method for the particular challenge you’re tasked with.

I want to...

• Inspect traffic from my browser.

  Use TLS key log to inspect traffic from Firefox, Chrome and curl. Use a TLS inspection proxy for other browsers.

• Inspect traffic to my HTTPS website.

  Use RSA private key inspection if it is acceptable to use an older TLS version and less secure ciphers. Use a TLS key log if your web server can be configured to export one or if you have an agent based key extraction solution that supports the TLS library used by the web server. Use a TLS inspection proxy if you want to inspect the traffic with something other than Wireshark.

• Inspect potentially malicious TLS traffic with an IDS or security appliance.

  Use a TLS inspection proxy.

• Inspect traffic from my operating system.

  Use a TLS inspection proxy.

• Inspect traffic from my mobile phone, smart device or other embedded device.

  Use a TLS inspection proxy.

• Inspect traffic from a proprietary game, app or service.

  Use a TLS inspection proxy.

Posted by Erik Hjelmvik on Wednesday, 07 August 2024 11:40:00 (UTC/GMT)

Tags: #TLS​ #TLS Inspection​ #PolarProxy​ #SSLKEYLOGFILE​ #Wireshark​ #PCAP​

PolarProxy 0.9 Released

PolarProxy was previously designed to only run as a transparent TLS proxy. But due to popular demand we’ve now extended PolarProxy to also include a SOCKS proxy and a HTTP CONNECT proxy. PolarProxy automatically decrypts all proxied SSL and TLS traffic, regardless if the remote server is running on TCP 443 or some other port, as long as the traffic passes through PolarProxy. As from now we also release a Windows build of PolarProxy, alongside the Linux x64, ARM and ARM64 builds.

SOCKS Proxy

Use the command line argument “--socks [port]” to start PolarProxy’s SOCKS proxy server. This SOCKS proxy supports multiple versions of the SOCKS protocol, including SOCKS 4, SOCKS 4a, SOCKS 5 and SOCKS 5h.

As an example, the command below starts a SOCKS server on TCP port 1080 and passes a copy of the decrypted TLS traffic as a PCAP stream to tshark.

You can then use curl to run some HTTPS traffic through the SOCKS proxy:

After doing this you should see the decrypted HTTP/2 traffic in tshark’s output.

HTTP CONNECT Proxy

We’ve also added a HTTP proxy to PolarProxy 0.9, but it only supports the CONNECT request method. This means that normal unencrypted HTTP requests, like GET or POST requests, will be rejected by PolarProxy. Most web traffic is TLS encrypted nowadays anyway, so we don't consider this limitation to be a big issue.

The HTTP CONNECT proxy service is activated with the “--httpconnect” argument. Decrypted TLS traffic from PolarProxy’s HTTP CONNECT proxy can be forwarded to tshark just like in the SOCKS example, but the traffic from these proxies can also be accessed through PCAP-over-IP like this:

You can then connect to PolarProxy’s PCAP-over-IP service with NetworkMiner by clicking File, Receive PCAP over IP, select “Connect to IP/port”, enter “localhost” and click the “Start Receiving” button. You’ll now be able to see a real-time feed of all the traffic that PolarProxy decrypts. As an example, let’s download the PolarProxy logo over HTTPS to see if NetworkMiner can extract it from PolarProxy’s decrypted PCAP-over-IP stream:

The PolarProxy logo immediately shows up in NetworkMiner’s images tab:

Port-Independent TLS Protocol Detection

When PolarProxy is running as a transparent TLS proxy all incoming traffic can be expected to be TLS. But that’s not the case when, for example, PolarProxy is running as a SOCKS proxy. We have therefore added port-independent TLS protocol detection for proxied traffic, so that TLS traffic can be detected and decrypted even when it runs on other ports than the standard 443, 465, 853, 990, 993, 995 and 5061 ones.

There is one crucial limitation to the automatic SSL/TLS protocol detection though, it doesn’t support explicit TLS traffic that relies on opportunistic encryption features like STARTTLS, which bootstraps TLS into an already established application layer session.

Allow Non-TLS Traffic

SOCKS and HTTP CONNECT proxies can both be used to transport other protocols than TLS. PolarProxy blocks all non-TLS traffic by default, but this setting can be overridden with the “--nontls allow” argument to allow any traffic to be proxied. The allow non-TLS override has no effect on PolarProxy’s transparent proxy though, because it will need to see a valid SNI field in order to know whereto the traffic should be forwarded.

Windows Build

There wasn’t much need for a Windows build of PolarProxy prior to the release of version 0.9, because the Windows firewall can’t be configured to redirect outgoing port 443 traffic to a local service. However, now that PolarProxy also includes SOCKS and HTTP CONNECT services, the situation is completely different. There are many ways to configure a Windows PC, as well as web browsers and other applications, to use a local proxy server.

You can use the Proxy settings window in Windows 10 and 11 to enable a local HTTP proxy like this:

Another option is to run “inetcpl.cpl” (Internet Options), open the “Connections” tab and click the “LAN settings” button to configure an HTTP proxy.

You can, of course, also configure your browser to use a local SOCKS or HTTP proxy in Windows, just as you’d do on any other operating system.

But don’t forget to configure your OS and/or browser to trust your PolarProxy instance’s root CA certificate first, as explained in the “Trusting the PolarProxy root CA” section of our PolarProxy documentation.

The Windows version of PolarProxy is a .NET framework-dependent application, which requires the .NET 6 runtime to be installed. The PolarProxy releases for other platforms (Linux x64, ARM and ARM64) are all self-contained applications, which are published with the .NET runtime built-in.

Visit our PolarProxy page to download and install PolarProxy.

Posted by Erik Hjelmvik on Thursday, 13 January 2022 10:15:00 (UTC/GMT)

Tags: #PolarProxy​ #proxy​ #SOCKS​ #SOCKS5​ #TLS​ #SSL​ #decrypt​ #Windows​ #PCAP-over-IP​ #pcapoverip​

Capturing Decrypted TLS Traffic with Arkime

The latest version of Arkime (The Sniffer Formerly Known As Moloch) can now be fed with a real-time stream of decrypted HTTPS traffic from PolarProxy. All that is needed to enable this feature is to include "pcapReadMethod=pcap-over-ip-server" in Arkime's config.ini file and start PolarProxy with the "--pcapoveripconnect 127.0.0.1:57012" option. PolarProxy will then connect to Arkime's PCAP-over-IP listener on TCP port 57012 and send it a copy of all TLS packets it decrypts.

Note: The required PCAP-over-IP feature is available in Arkime 2.7.0 and PolarProxy 0.8.16.

Arkime is an open source packet capture solution that indexes the PCAP data it collects. Arkime also comes with a web frontend for browsing and searching through the captured, and indexed, network traffic. The Arkime project recently changed name from Moloch, probably in an attempt to convince users that the tool doesn't eat children.

This guide demonstrates how TLS traffic, or more specifically HTTPS traffic, can be decrypted and ingested in real-time into Arkime.

The TLS decryption is performed with PolarProxy, which is a transparent TLS interception proxy that is freely available under a Creative Commons BY-ND 4.0 license.

PolarProxy and Arkime can be installed on a server to intercept, decrypt, index and store decrypted TLS network traffic from multiple clients on a network. It is even possible to install PolarProxy and Arkime on separate servers, so that PolarProxy forwards a stream of decrypted traffic to the Arkime server. However, to avoid unnecessary complexity, Arkime and PolarProxy are installed locally on a Linux client in this howto guide. The Linux client is a Ubuntu 20.04.1 machine, but the instructions can also be used on other Linux flavors that use systemd, such as Arch, CentOS, Debian, Fedora, SUSE and Red Hat Linux.

Download and Install Arkime

Arkime can be downloaded as a pre-built installation packages for CentOS and Ubuntu here: https://arkime.com/#download

Note: You can alternatively visit the Arkime GitHub page if there is no pre-built installation package for your Linux distro or you prefer to build Arkime from source.

After installing the Arkime package, configure Arkime by running:

• Enter "none" as the interface to monitor (the interface setting will be ignored when Arkime gets configured as a PCAP-over-IP server)
• Install the ElasticSearch server by typing "yes" when prompted

Edit /data/moloch/etc/config.ini and add "pcapReadMethod=pcap-over-ip-server" to configure Arkime to listen for PCAP-over-IP connections.

Next, enable and start the ElasticSearch systemd service.

Initiate the Arkime search cluster.

Create a new admin user.

You can now enable and start the Moloch capture and viewer services.

Verify that Arkime now listens for incoming connections on TCP port 57012.

Install PolarProxy to Decrypt TLS Traffic

Create a user for PolarProxy's systemd service and download PolarProxy like this:

Copy the default PolarProxy service config to the systemd location.

Modify /etc/systemd/system/PolarProxy.service by adding "--pcapoveripconnect 127.0.0.1:57012" at the end of the ExecStart command.

It's now time to enable and start the PolarProxy service.

Verify that PolarProxy has connected to Arkime's PCAP-over-IP listener on TCP port 57012.

Take it For a Test Run

PolarProxy is listening for incoming TLS connections on TCP port 10443. We can therefore run traffic through the TLS decryption proxy with this curl command:

The decrypted traffic will show up in Arkime if everything is working. Open http://localhost:8005/sessions in a browser and look for a connection to www.netresec.com.

Note: The Arkime username and password is admin/THEPASSWORD if you've followed the instructions in this tutorial.

Also: You might have to wait a minute or two for the traffic to appear in Arkime's user interface.

Trust PolarProxy's Root CA Certificate

The root CA certificate used by your PolarProxy service must be trusted by both the operating system and browser in order to run TLS traffic through the decryption proxy without errors. Follow these instructions to add trust the root CA:

• Select the "extra/PolarProxy-root-CA.crt" Certificate Authority
• Press <Ok>

Start Firefox

• Download the root CA certificate from: http://localhost:10080/polarproxy.cer
• Open: about:preferences#privacy
• Scroll down to "Certificates" and click "View Certificates"
• Import > Select "polarproxy.cer"
• Select: ☑ Trust this CA to identify websites

Configure Firewall Redirect of Outgoing HTTPS Traffic

The final step in this tutorial is to redirect the local user's outgoing HTTPS traffic to the PolarProxy service listening on TCP port 10443. Add the following lines at the top of /etc/ufw/before.rules (before the "*filter" section) to redirect outgoing HTTPS traffic to the local PolarProxy service listening on port 10443.

Note: The UFW config in "before.rules" is equivalent to running "iptables -t nat -A OUTPUT -m owner --uid 1000 -p tcp --dport 443 -j REDIRECT --to 10443"

Make sure to modify the uid value (1000) in the firewall rule to match that of the local user that PolarProxy should decrypt the HTTPS traffic for. You can see your uid value by running the command "id -u". You can even redirect traffic from several users to PolarProxy, but it's important that you DON'T forward the outgoing HTTPS traffic from the "proxyuser" account. You will otherwise generate an infinite firewall redirect loop, where outgoing HTTPS traffic from PolarProxy is redirected back to PolarProxy again. You can check the proxyuser's uid with the command "id -u proxyuser".

After saving before.rules, reload UFW to activate the port redirection.

Surf 'n' Snoop

Your Linux machine is now configured to send decrypted HTTPS traffic to Arkime for inspection. Open Firefox and visit some websites, then go back to Arkime and have a look at the traffic. Again, remember that there might be a few minutes' delay before the traffic appears in Arkime's user interface

You'll probably notice that the majority of all HTTPS traffic is actually using the HTTP/2 protocol. Unfortunately Arkime's http2 support is still quite limited, but I'm hoping it will improve in future releases.

Luckily, both Wireshark and NetworkMiner (which runs fine in Linux by the way) can be used to parse and extract contents from HTTP/2 traffic. Just hit Arkime's "Download PCAP" button and open the capture file in a tool of your choice.

Image: NetworkMiner in Linux with files extracted from decrypted HTTP/2 traffic

Posted by Erik Hjelmvik on Tuesday, 01 December 2020 07:50:00 (UTC/GMT)

Tags: #Arkime​ #PolarProxy​ #TLS​ #HTTPS​ #decrypt​ #PCAP​ #systemd​ #systemctl​ #UFW​ #http2​ #HTTP/2​ #PCAP-over-IP​ #pcapoverip​ #ASCII-art​

Discovered Artifacts in Decrypted HTTPS

We released a PCAP file earlier this year, which was recorded as part of a live TLS decryption demo at the CS3Sthlm conference. The demo setup used PolarProxy running on a Raspberry Pi in order to decrypt all HTTPS traffic and save it in a PCAP file as unencrypted HTTP.

This capture file was later used as a challenge for our twitter followers, when we made the following announcement:

PCAP CHALLENGE!
The capture file released in this blog post contains a few interesting things that were captured unintentionally. Can you find anything strange, funny or unexpected in the pcap file? (1/2)

Followed by this message:

The person to submit the most interesting answer wins a “PCAP or it didn’t happen” t-shirt. Compete by including your discovery in a retweet or reply to this tweet, or in an email to info(at)netresec.com. We want your answers before the end of January. (2/2)

We'd like to thank everyone who submitted answers in this challenge, such as David Ledbetter, Christoffer Strömblad, RunΞ and Chris Sistrunk.

We're happy to announce that the winner of our challenge is David Ledbetter. Congratulations David!

So what were the interesting thing that could be found in the released capture file? Below is a short summary of some things that can be found.

Telemetry data sent to mozilla.org

A surprising amount of information about the Firefox browser was sent to incoming.telemetry.mozilla.org, including things like:

• Active browser addons
• Active browser plugins
• Firefox profile creation date
• Browser search region
• Default search engine
• Regional locales
• Screen width
• Screen height
• CPU vendor, family and model
• HDD model, revision and type
• Installed RAM
• Operating system
• Etc..

Here's an excerpt showing a part of the data sent to Mozilla:

"build": { "applicationId": "{ec8030f7-c20a-464f-9b0e-13a3a9e97384}", "applicationName": "Firefox", "architecture": "x86-64", "buildId": "20191002194346", "version": "69.0.2", "vendor": "Mozilla", "displayVersion": "69.0.2", "platformVersion": "69.0.2", "xpcomAbi": "x86_64-gcc3", "updaterAvailable": false }, "partner": { "distributionId": "canonical", "distributionVersion": "1.0", "partnerId": "ubuntu", "distributor": "canonical", "distributorChannel": "ubuntu", "partnerNames": [ "ubuntu" ] }, "system": { "memoryMB": 3943, "virtualMaxMB": null, "cpu": { "count": 1, "cores": 1, "vendor": "GenuineIntel", "family": 6, "model": 42, "stepping": 7, "l2cacheKB": 256, "l3cacheKB": 4096, "speedMHz": null, "extensions": [ "hasMMX", "hasSSE", "hasSSE2", "hasSSE3", "hasSSSE3", "hasSSE4_1", "hasSSE4_2", "hasAVX", "hasAES" ] }, "os": { "name": "Linux", "version": "5.0.0-31-generic", "locale": "en-US" }, "hdd": { "profile": { "model": null, "revision": null, "type": null }, "binary": { "model": null, "revision": null, "type": null }, "system": { "model": null, "revision": null, "type": null } }, "gfx": { "D2DEnabled": null, "DWriteEnabled": null, "ContentBackend": "Skia", "Headless": false, "adapters": [ { "description": "llvmpipe (LLVM 8.0, 256 bits)", "vendorID": "0xffff", "deviceID": "0xffff", "subsysID": null, "RAM": 3942, "driver": null, "driverVendor": "mesa/llvmpipe", "driverVersion": "19.0.8.0", "driverDate": null, "GPUActive": true } ], "monitors": [ { "screenWidth": 681, "screenHeight": 654 } ], "features": { "compositor": "basic", "gpuProcess": { "status": "unavailable" }, "wrQualified": { "status": "blocked-vendor-unsupported" }, "webrender": { "status": "opt-in" } } }, "appleModelId": null }, "settings": { "blocklistEnabled": true, "e10sEnabled": true, "e10sMultiProcesses": 8, "telemetryEnabled": false, "locale": "en-US", "intl": { "requestedLocales": [ "en-US" ], "availableLocales": [ "en-US", "en-CA", "en-GB" ], "appLocales": [ "en-US", "en-CA", "en-GB", "und" ], "systemLocales": [ "en-US" ], "regionalPrefsLocales": [ "sv-SE" ], "acceptLanguages": [ "en-US", "en" ] }, "update": { "channel": "release", "enabled": true, "autoDownload": false }, "userPrefs": { "browser.cache.disk.capacity": 1048576, "browser.search.region": "SE", "browser.search.widget.inNavBar": false, "network.trr.mode": 2 }, "sandbox": { "effectiveContentProcessLevel": 4 }, "addonCompatibilityCheckEnabled": true, "isDefaultBrowser": false, "defaultSearchEngine": "google", "defaultSearchEngineData": { "name": "Google", "loadPath": "[distribution]/searchplugins/locale/en-US/google.xml", "origin": "default", "submissionURL": "https://www.google.com/search?client=ubuntu&channel=fs&q=&ie=utf-8&oe=utf-8" } }, "profile": { "creationDate": 18183, "firstUseDate": 18183 }

You can use the following Wireshark display filter to find all the data sent to Mozilla:

http.request.method eq POST and http.host contains telemetry

Public IP Revealed in PCAP

The client's IP address was 192.168.4.20, which is part of the RFC 1918 192.168/16 private address space. It's therefore safe to assume that the client was behind a NAT (the client was in fact behind a double NAT). However, we noticed that the public IP of the client was revealed through multiple services in the captured network traffic. One of these services is the advertising exchange company AppNexus (adnxs.com), which sent the client's public IP address 193.235.19.252 in an X-Proxy-Origin HTTP header.

You can use the following Wireshark/tshark display filter to find X-Proxy-Origin headers:

http.response.line matches "x-proxy-origin" or http2.header.name matches "x-proxy-origin"

We are using the "matches" operator here instead of "contains" or "==" because we want to perform case insensitive matching. You might also notice that we need a completely different display filter syntax to match HTTP/2 headers compared to what we are used to with HTTP/1.1.

Monty Python "Majestik møøse" reference in reddit x-header

The reddit server 151.101.85.140 sends an HTTP/2 header called "x-moose" with a value of "majestic".

This header refers to the opening credits of Monty Python and the Holy Grail.

Wi nøt trei a høliday in Sweden this yër?

Posted by Erik Hjelmvik on Tuesday, 17 March 2020 09:00:00 (UTC/GMT)

Tags: #HTTP/2​ #http2​ #TLS​ #decrypt​ #TLSI​ #PolarProxy​ #NetworkMiner​ #Wireshark​ #CS3Sthlm​ #CS3​ #Forensics​ #PCAP​ #ASCII-art​ #RFC1918​

Reverse Proxy and TLS Termination

PolarProxy is primarily a TLS forward proxy, but it can also be used as a TLS termination proxy or reverse TLS proxy to intercept and decrypt incoming TLS traffic, such as HTTPS or IMAPS, before it is forwarded to a server. The proxied traffic can be accessed in decrypted form as a PCAP formatted data stream, which allows real-time analysis of the decrypted traffic by an IDS as well as post incident forensics with Wireshark.

PolarProxy version 0.8.15 and later can import an existing X.509 server certificate (aka leaf certificate or end-entity certificate) in order to perform the TLS decryption using a valid certificate signed by a trusted certificate authority. If no server certificate is provided, then PolarProxy falls back to generating server certificates on the fly and signing them with its own root CA certificate.

There are two principal ways to run PolarProxy as a reverse proxy, either as a TLS termination proxy or as a reverse proxy that decrypts and re-encrypts the traffic.

PolarProxy as a TLS Termination Proxy

The TLS termination proxy mode is useful in order to offload the task of performing TLS encryption to PolarProxy instead of doing the decryption on the web server. This mode can also be used when the proxied services don’t support TLS encryption, such as legacy web servers or servers hosting other unencrypted services that you want to secure with TLS.

The following command sequence shows how to create a Let’s Encrypt SSL certificate, convert it to the PKCS#12 format, and load the server certificate into PolarProxy to terminate incoming HTTPS connections. In this setup PolarProxy decrypts the TLS traffic and relays the HTTP traffic to the web server on TCP port 80.

Here’s a breakdown of the arguments sent to PolarProxy:

• --terminate : Terminate incoming TLS sessions and forward proxied traffic in unencrypted form.
• --connect 10.1.2.3 : Forward all proxied traffic to 10.1.2.3 instead of connecting to the host name provided in the SNI extension of the TLS ClientHello message.
• --nosni www.example.com : Treat incoming TLS sessions that don’t define a host name with the SNI extension as if they wanna to connect to “www.example.com”.
• --leafcert load:example.com,www.example.com:/etc/example.p12:PASSWORD : Use the server certificate “/etc/example.p12” for incoming connections to “example.com” and “www.example.com”.
• -p 443,80,80 : Listen on TCP port 443, save decrypted traffic in PCAP file as if it was directed to port 80, forward decrypted traffic to port 80.
• -o /var/log/TlsTerminationProxy/ : Save decrypted traffic to hourly rotated PCAP files in “/var/log/TlsTerminationProxy/”.

PolarProxy is a generic TLS proxy that doesn’t care what application layer protocol the TLS tunnel carries. So if you want to terminate the TLS encryption of incoming IMAPS sessions as well, then simply append an additional argument saying “-p 993,143,143” to also forward decrypted IMAP sessions to 10.1.2.3. This method can be used in order to wrap almost any TCP based protocol in a TLS tunnel, which can be useful for privacy reasons as well as to prevent network monitoring tools from detecting the actual application layer protocol.

PolarProxy as a Reverse TLS Proxy

There are setups for which it is preferable to also encrypt the internal sessions between PolarProxy and the final server. One such setup is when the server is hosting a web service with support for the HTTP/2 protocol, which in practice always uses TLS. Luckily PolarProxy is designed to decrypt and re-encrypt proxied traffic while also forwarding important TLS parameters, such as ALPN and SNI, between the internal and external TLS sessions.

To use TLS encryption on the inside as well as outside of PolarProxy, simply do as explained in the previous TLS termination section, but remove the “--terminate” argument and change the port argument to “-p 443,80,443” like this:

PolarProxy will save the decrypted traffic as cleartext HTTP (or HTTP/2) to PCAP files in the “/var/log/ReverseTlsProxy/” directory.

Real-Time Analysis of Decrypted Traffic

Both the external (client-to-proxy) and internal (proxy-to-server) TCP sessions, in the reverse TLS proxy example above, are encrypted with TLS. This prevents passive network security monitoring tools, such as IDSs, DPI and DLP appliances, from analyzing the application layer data being sent and received. The PCAP files written to “/var/log/ReverseTlsProxy/” can be a valuable forensic asset when investigating an incident, but a real-time stream of the decrypted data is needed in order to swiftly detect and alert on potential security breaches and other incidents.

PolarProxy’s “--pcapoverip” option can be used to provide such a real-time stream of the decrypted data passing through the proxy. This data can easily be sent to a network interface using tcpreplay, as explained in our blog post “Sniffing Decrypted TLS Traffic with Security Onion”.

Security Considerations

The examples shown in this blog post all run PolarProxy with root privileges using sudo, which can be dangerous from a security perspective. PolarProxy is actually designed to be run without root privileges, but doing so prevents it from listening on a port below 1024. Luckily, this issue can easily be overcome with a simple port forwarding or redirect rule. The following iptables redirect rule can be used if PolarProxy is listening on TCP port 20443 and incoming HTTPS request are arriving to the eth0 interface of the proxy:

PolarProxy does not support loading settings from a config file. The password for the PKCS12 certificate will therefore need to be supplied on the command line, which can make it visible from a process listing. If this is a concern for you, then please consider using “hidepid” to hide processes from other users. You can find instructions on how to use hidepid in hardening guides for Debian, Arch, SUSE and most other Linux flavors.

Posted by Erik Hjelmvik on Thursday, 12 March 2020 15:45:00 (UTC/GMT)

Tags: #PolarProxy​ #TLS​ #SSL​ #PCAP​ #decrypt​ #HTTPS​ #HTTP​ #HTTP/2​ #http2​ #IMAPS​ #SNI​ #decrypt​ #ASCII-art​

Sharing a PCAP with Decrypted HTTPS

Modern malware and botnet C2 protocols use TLS encryption in order to blend in with "normal" web traffic, sometimes even using legitimate services like Twitter or Instagram.

I did a live demo at the CS3Sthlm conference last year, titled "TLS Interception and Decryption", where I showed how TLS interception can be used to decrypt and analyze malicious HTTPS network traffic. During the demo I used DNS-over-HTTPS (DoH) and posted messages to Pastebin and Twitter, pretending to be a malware or malicious actor. The HTTPS network traffic was decrypted and analyzed live as part of my demo. The CS3Sthlm organizers have posted a video recording of the live demo on YouTube.

Image: Erik demoing TLS Interception and Decryption at CS3Sthlm 2019

We are now releasing a PCAP file with the decrypted network traffic captured during this live demo here:

» https://media.netresec.com/pcap/proxy-191023-091924.pcap «

This blog post provides a step-by-step walk-through of the decrypted HTTPS traffic in the released capture file.

The TLS decryption was performed by connecting a laptop to a custom WiFi access point, which was a Raspberry Pi configured as in our "Raspberry Pi WiFi Access Point with TLS Inspection" blog post. I additionally enabled the PCAP-over-IP feature in PolarProxy by starting it with the "--pcapoverip 57012" option. This allowed me to connect with Wireshark and NetworkMiner to TCP port 57012 on the TLS proxy and stream the decrypted traffic in order to perform live network traffic analysis.

Image: Live demo network with Laptop (Browser, NetworkMiner, Wireshark), Raspberry Pi (PolarProxy) and the Internet.

Below follows a breakdown of various significant events of my demo and where you can find these events in the released capture file.

DNS lookup of "www.google.com" using DoH

• Frame: 833
• Protocol: DoH using HTTP/2 POST
• Five tuple: 192.168.4.20:52694 104.16.248.249:80 TCP

Google search for "tibetan fox psbattle"

• Frame: 2292
• Protocol: HTTP/2
• Five tuple: 192.168.4.20:52716 216.58.211.4:80 TCP

Tibetan Fox image downloaded from reddit

• Frame: 3457
• Protocol: HTTP/2
• Five tuple: 192.168.4.20:52728 151.101.85.140:80 TCP

Orginal "tibetan fox" image downloaded from this reddit thread.

Tibetan Fox Remix Image HTTP/2 Download

• Frame: 5805
• Protocol: HTTP/2
• Five tuple: 192.168.4.20:52769 151.101.84.193:80 TCP

DNS Lookup of "cs3sthlm.se"

• Frame: 13494
• Protocol: DoH using HTTP/2 POST
• Five tuple: 192.168.4.20:52699 104.16.249.249:80 TCP

Images downloaded from CS3Sthlm's website

• Frame: 14134
• Protocol: HTTP/1.1
• Five tuple: 192.168.4.20:52896 192.195.142.160:80 TCP

Data sent in HTTP/2 POST to Pastebin

• Frame: 18572
• Protocol: HTTP/2 POST
• Five tuple: 192.168.4.20:52904 104.22.2.84:80 TCP

The file "post.php.form-data" contains the data sent to Pastebin in the HTTP/2 POST request. Here are the reassembled contents of that file, including the "hello cs3 I am a malware" message:

Mallory80756920 logs in to Twitter

• Frame: 24881
• Protocol: HTTP/2 POST
• Five tuple: 192.168.4.20:53210 104.244.42.65:80 TCP

Mallory80756920 posts a Tweet

• Frame: 26993
• Protocol: HTTP/2 POST
• Five tuple: 192.168.4.20:53251 104.244.42.66:80 TCP

Mallory80756920 tweeted "Hello CS3! I'm in you!". The data was sent to twitter using an HTTP/2 POST request.

Conclusions

A great deal of the interesting TLS traffic in the analyzed capture file is using the HTTP/2 protocol. This doesn't come as a surprise since more than half of all HTTPS traffic is using HTTP/2 nowadays (sources: server protocol statistics, client protocol statistics). It is therefore essential to be able to analyze HTTP/2 traffic if you have a TLS inspection (TLSI) solution in place. Unfortunately many TLSI products don't yet support the HTTP/2 protocol.

Wireshark was one of the first network traffic analysis tools to implement HTTP/2 support, much thanks to Alexis La Goutte. However, Wireshark's excellent "File > Export Objects" doesn't yet support extraction of files from HTTP/2 traffic. There are other ways to extract HTTP/2 file transfers with Wireshark, but they require a few additional steps in order to carve out the file to disk.

Luckily NetworkMiner extracts files from HTTP/2 as of version 2.5. In fact, we believe NetworkMiner is the first open source tool to support automatic HTTP/2 file extraction from PCAP.

Finally, I'd like to stress the point that modern malware use HTTPS, so you need to have a TLSI solution in place to analyze the malicious traffic. As the majority of all HTTPS traffic is using HTTP/2 you also need to ensure that you're able to analyze HTTP/2 traffic passing through your TLSI solution.

Posted by Erik Hjelmvik on Monday, 13 January 2020 12:45:00 (UTC/GMT)

Tags: #HTTP/2​ #http2​ #DoH​ #TLS​ #Google​ #decrypt​ #HTTPS​ #TLSI​ #TLS Inspection​ #TLS Interception​ #PolarProxy​ #NetworkMiner​ #Wireshark​ #CS3Sthlm​ #CS3​ #Forensics​ #PCAP​ #Video​