Capturing Decrypted TLS Traffic with Arkime

The latest version of Arkime (The Sniffer Formerly Known As Moloch) can now be fed with a real-time stream of decrypted HTTPS traffic from PolarProxy. All that is needed to enable this feature is to include "pcapReadMethod=pcap-over-ip-server" in Arkime's config.ini file and start PolarProxy with the "--pcapoveripconnect 127.0.0.1:57012" option. PolarProxy will then connect to Arkime's PCAP-over-IP listener on TCP port 57012 and send it a copy of all TLS packets it decrypts.

Note: The required PCAP-over-IP feature is available in Arkime 2.7.0 and PolarProxy 0.8.16.

Arkime is an open source packet capture solution that indexes the PCAP data it collects. Arkime also comes with a web frontend for browsing and searching through the captured, and indexed, network traffic. The Arkime project recently changed name from Moloch, probably in an attempt to convince users that the tool doesn't eat children.

This guide demonstrates how TLS traffic, or more specifically HTTPS traffic, can be decrypted and ingested in real-time into Arkime.

The TLS decryption is performed with PolarProxy, which is a transparent TLS interception proxy that is freely available under a Creative Commons BY-ND 4.0 license.

PolarProxy and Arkime can be installed on a server to intercept, decrypt, index and store decrypted TLS network traffic from multiple clients on a network. It is even possible to install PolarProxy and Arkime on separate servers, so that PolarProxy forwards a stream of decrypted traffic to the Arkime server. However, to avoid unnecessary complexity, Arkime and PolarProxy are installed locally on a Linux client in this howto guide. The Linux client is a Ubuntu 20.04.1 machine, but the instructions can also be used on other Linux flavors that use systemd, such as Arch, CentOS, Debian, Fedora, SUSE and Red Hat Linux.

Download and Install Arkime

Arkime can be downloaded as a pre-built installation packages for CentOS and Ubuntu here: https://arkime.com/#download

Note: You can alternatively visit the Arkime GitHub page if there is no pre-built installation package for your Linux distro or you prefer to build Arkime from source.

After installing the Arkime package, configure Arkime by running:

• Enter "none" as the interface to monitor (the interface setting will be ignored when Arkime gets configured as a PCAP-over-IP server)
• Install the ElasticSearch server by typing "yes" when prompted

Edit /data/moloch/etc/config.ini and add "pcapReadMethod=pcap-over-ip-server" to configure Arkime to listen for PCAP-over-IP connections.

Next, enable and start the ElasticSearch systemd service.

Initiate the Arkime search cluster.

Create a new admin user.

You can now enable and start the Moloch capture and viewer services.

Verify that Arkime now listens for incoming connections on TCP port 57012.

Install PolarProxy to Decrypt TLS Traffic

Create a user for PolarProxy's systemd service and download PolarProxy like this:

Copy the default PolarProxy service config to the systemd location.

Modify /etc/systemd/system/PolarProxy.service by adding "--pcapoveripconnect 127.0.0.1:57012" at the end of the ExecStart command.

It's now time to enable and start the PolarProxy service.

Verify that PolarProxy has connected to Arkime's PCAP-over-IP listener on TCP port 57012.

Take it For a Test Run

PolarProxy is listening for incoming TLS connections on TCP port 10443. We can therefore run traffic through the TLS decryption proxy with this curl command:

The decrypted traffic will show up in Arkime if everything is working. Open http://localhost:8005/sessions in a browser and look for a connection to www.netresec.com.

Note: The Arkime username and password is admin/THEPASSWORD if you've followed the instructions in this tutorial.

Also: You might have to wait a minute or two for the traffic to appear in Arkime's user interface.

Trust PolarProxy's Root CA Certificate

The root CA certificate used by your PolarProxy service must be trusted by both the operating system and browser in order to run TLS traffic through the decryption proxy without errors. Follow these instructions to add trust the root CA:

• Select the "extra/PolarProxy-root-CA.crt" Certificate Authority
• Press <Ok>

Start Firefox

• Download the root CA certificate from: http://localhost:10080/polarproxy.cer
• Open: about:preferences#privacy
• Scroll down to "Certificates" and click "View Certificates"
• Import > Select "polarproxy.cer"
• Select: ☑ Trust this CA to identify websites

Configure Firewall Redirect of Outgoing HTTPS Traffic

The final step in this tutorial is to redirect the local user's outgoing HTTPS traffic to the PolarProxy service listening on TCP port 10443. Add the following lines at the top of /etc/ufw/before.rules (before the "*filter" section) to redirect outgoing HTTPS traffic to the local PolarProxy service listening on port 10443.

Note: The UFW config in "before.rules" is equivalent to running "iptables -t nat -A OUTPUT -m owner --uid 1000 -p tcp --dport 443 -j REDIRECT --to 10443"

Make sure to modify the uid value (1000) in the firewall rule to match that of the local user that PolarProxy should decrypt the HTTPS traffic for. You can see your uid value by running the command "id -u". You can even redirect traffic from several users to PolarProxy, but it's important that you DON'T forward the outgoing HTTPS traffic from the "proxyuser" account. You will otherwise generate an infinite firewall redirect loop, where outgoing HTTPS traffic from PolarProxy is redirected back to PolarProxy again. You can check the proxyuser's uid with the command "id -u proxyuser".

After saving before.rules, reload UFW to activate the port redirection.

Surf 'n' Snoop

Your Linux machine is now configured to send decrypted HTTPS traffic to Arkime for inspection. Open Firefox and visit some websites, then go back to Arkime and have a look at the traffic. Again, remember that there might be a few minutes' delay before the traffic appears in Arkime's user interface

You'll probably notice that the majority of all HTTPS traffic is actually using the HTTP/2 protocol. Unfortunately Arkime's http2 support is still quite limited, but I'm hoping it will improve in future releases.

Luckily, both Wireshark and NetworkMiner (which runs fine in Linux by the way) can be used to parse and extract contents from HTTP/2 traffic. Just hit Arkime's "Download PCAP" button and open the capture file in a tool of your choice.

Image: NetworkMiner in Linux with files extracted from decrypted HTTP/2 traffic

Posted by Erik Hjelmvik on Tuesday, 01 December 2020 07:50:00 (UTC/GMT)

Tags: #Arkime​ #PolarProxy​ #TLS​ #HTTPS​ #decrypt​ #PCAP​ #systemd​ #systemctl​ #UFW​ #http2​ #HTTP/2​ #PCAP-over-IP​ #pcapoverip​ #ASCII-art​

Sniffing Decrypted TLS Traffic with Security Onion

Wouldn't it be awesome to have a NIDS like Snort, Suricata or Zeek inspect HTTP requests leaving your network inside TLS encrypted HTTPS traffic? Yeah, we think so too! We have therefore created this guide on how to configure Security Onion to sniff decrypted TLS traffic with help of PolarProxy.

PolarProxy is a forward TLS proxy that decrypts incoming TLS traffic from clients, re-encrypts it and forwards it to the server. One of the key features in PolarProxy is the ability to export the proxied traffic in decrypted form using the PCAP format (a.k.a. libpcap/tcpdump format). This makes it possible to read the decrypted traffic with external tools, without having to perform the decryption again. It also enables packet analysis using tools that don't have built-in TLS decryption support.

This guide outlines how to configure PolarProxy to intercept HTTPS traffic and send the decrypted HTTP traffic to an internal network interface, where it can be sniffed by an IDS.

STEP 1 ☆ Install Ubuntu

Download and install the latest SecurityOnion ISO image, but don't run the "Setup" just yet.

STEP 2 ☆ Add a Dummy Network Interface

Add a dummy network interface called "decrypted", to which decrypted packets will be sent.

STEP 3 ☆ Install Updates

Install updates in Security Onion by running "sudo soup".

STEP 4 ☆ Run the Security Onion Setup

Run the Security Onion setup utility by double-clicking the "Setup" desktop shortcut or executing "sudo sosetup" from a terminal. Follow the setup steps in the Production Deployment documentation and select "decrypted" as your sniffing interface.

Reboot and run Setup again to continue with the second phase of Security Onion's setup. Again, select "decrypted" as the interface to be monitored.

STEP 5 ☆ Install PolarProxy Service

Download and install PolarProxy:

Edit /etc/systemd/system/PolarProxy.service and add "--pcapoverip 57012" at the end of the ExecStart command.

Start the PolarProxy systemd service:

STEP 6 ☆ Install Tcpreplay Service

The decrypted traffic can now be accessed via PolarProxy's PCAP-over-IP service on TCP 57012. We can leverage tcpreplay and netcat to replay these packets to our dummy network interface in order to have them picked up by Security Onion.

Start the tcpreplay systemd service:

STEP 7 ☆ Add firewall rules

Security Onion only accepts incoming connections on TCP 22 by default, we also need to allow connections to TCP port 10443 (proxy port), and 10080 (root CA certificate download web server). Add allow rules for these services to the Security Onion machine's firewall:

Verify that the proxy is working by running this curl command on a PC connected to the same network as the Security Onion machine:

Add the following lines at the top of /etc/ufw/before.rules (before the *filter section) to redirect incoming packets on TCP 443 to PolarProxy on port 10443.

Note: Replace "enp0s3" with the Security Onion interface to which clients will connect.

After saving before.rules, reload ufw to activate the port redirection:

Verify that you can reach the proxy on TCP 443 with this command:

STEP 8 ☆ Redirect HTTPS traffic to PolarProxy

It's now time to configure a client to run its HTTPS traffic through PolarProxy. Download and install the PolarProxy X.509 root CA certificate from PolarProxy's web service on TCP port 10080:

http://[SecurityOnionIP]:10080/polarproxy.cer

Install the certificate in the operating system and browser, as instructed in the PolarProxy documentation.

You also need to forward packets from the client machine to the Security Onion machine running PolarProxy. This can be done either by configuring a local NAT rule on each monitored client (STEP 8.a) or by configuring the default gateway's firewall to forward HTTPS traffic from all clients to the proxy (STEP 8.b).

STEP 8.a ☆ Local NAT

Use this firewall rule on a Linux client to configure it to forward outgoing HTTPS traffic to the Security Onion machine:

STEP 8.b ☆ Global NAT

If the client isn't running Linux, or if you wanna forward HTTPS traffic from a whole network to the proxy, then apply the following iptables rules to the firewall in front of the client network. See "Routing Option #2" in the PolarProxy documentation for more details.

1. Add a forward rule on the gateway to allow forwarding traffic to our PolarProxy server:
   sudo iptables -A FORWARD -i eth1 -d [SecurityOnionIP] -p tcp --dport 10443 -m state --state NEW -j ACCEPT
2. Add a DNAT rule to forward 443 traffic to PolarProxy on port 10443:
   sudo iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j DNAT --to [SecurityOnionIP]:10443
3. If the reverse traffic from PolarProxy to the client doesn't pass the firewall (i.e. they are on the same LAN), then we must add this hide-nat rule to fool PolarProxy that we are coming from the firewall:
   sudo iptables -t nat -A POSTROUTING -o eth1 -d [SecurityOnionIP] -p tcp --dport 10443 -j MASQUERADE

STEP 9 ☆ Inspect traffic in SecurityOnion

Wait for the Elastic stack to initialize, so that the intercepted network traffic becomes available through the Kibana GUI. You can check the status of the elastic initialization with "sudo so-elastic-status".

You should now be able to inspect decrypted traffic in Security Onion using Kibana, Squert, Sguil etc., just as if it was unencrypted HTTP.

Image: Kibana showing HTTP traffic info from decrypted HTTPS sessions

Image: MIME types in Kibana

Image: NIDS alerts from payload in decrypted traffic shown in Kibana

Image: Snort alerts from decrypted traffic shown in Squert

Security Considerations and Hardening

Security Onion nodes are normally configured to only allow access by SOC/CERT/CSIRT analysts, but the setup described in this blog post requires that "normal" users on the client network can access the PolarProxy service running on the Security Onion node. We therefore recommend installing PolarProxy on a dedicated Security Onion Forward Node, which is configured to only monitor traffic from the proxy.

We also recommend segmenting the client network from the analyst network, for example by using separate network interfaces on the Security Onion machine or putting it in a DMZ. Only the PolarProxy service (TCP 10080 and 10443) should be accessable from the client network.

PolarProxy could be used to pivot from the client network into the analyst network or to access the Apache webserver running on the Security Onion node. For example, the following curl command can be used to access the local Apache server running on the Security Onion machine via PolarProxy:

Hardening Steps:

• Configure the Security Onion node as a Forward Node
• Segment client network from analyst network
• Add firewall rules to prevent PolarProxy from accessing services on the local machine and analyst network

For additional info on hardening, please see the recommendations provided by Wes Lambert on the Security-Onion mailing list.

Posted by Erik Hjelmvik on Monday, 20 January 2020 09:40:00 (UTC/GMT)

Tags: #SecurityOnion​ #Security Onion​ #PCAP​ #Bro​ #Zeek​ #PolarProxy​ #Snort​ #Suricata​ #TLS​ #SSL​ #HTTPS​ #tcpreplay​ #PCAP-over-IP​ #IDS​ #NIDS​ #netcat​ #curl​ #UFW​ #ASCII-art​