NetworkMiner 3.0 Released

I am very proud to announce the release of NetworkMiner 3.0 today!

This version brings several new protocols as well as user interface improvements to NetworkMiner. We have also made significant changes under the hood, such as altering the default location to where NetworkMiner extracts files from network traffic.

Some of the major changes in this new release are:

• New protocols: QUIC, CIP (EtherNet/IP), UMAS and Remcos RAT.
• Improved passive OS fingerprinting.
• Additional filtering capabilities.
• User interface adapted for Linux.

Filtering of Displayed Artefacts

A tooltip text is temporarily displayed when a filter is activated. The tooltip shows the number of visible items after the filter is applied. This tooltip can also be shown at a later point by hovering with the mouse over the filter text or the Apply button.

Right-clicking on an item or artefact in any of NetworkMiner’s tabs brings up a context menu. We’ve now added an “Apply as Filter” option to this context menu, which can be used to let NetworkMiner automatically generate a filter based on the clicked item. This feature saves time for the analyst and reduces risk of misspellings.

We have also added a keyword filter to the Credentials tab and updated the image filename filter to ignore case.

Other User Interface Improvements

The File Details window, which shows metadata and contents of an extracted file, now has a “Show as” menu that can be used to preview the contents of a file as ASCII, Hex, Unicode or UTF-8.

This file details window can now also be accessed directly from the Images tab by right-clicking on a thumbnail of an extracted image.

NetworkMiner 3.0 extracts Maximum Segment Size (MSS) values from TCP handshakes and show them under Host Details for each respective IP address. This value can help with determining if a host is behind a VPN. An MSS value below 1400 indicates that the traffic might have passed through some form of overlay network, such as a tunnel or VPN.

Other indicators that can help identify VPN and tunnelled traffic is IP TTL and latency, which NetworkMiner already extracts.

The screenshot above also shows that the operating system was identified as Windows, both with help of p0f as well as based on the client’s web browser user-agent. The user-agent based OS fingerprinting is a new feature that we added in NetworkMiner 3.0. This is a nice complement to the TCP and DHCP based OS fingerprinting features that NetworkMiner already performs. We’ve configured this feature to also detect operating systems of user-agent strings sent over UPnP/SSDP.

The text on a few of NetworkMiner’s buttons were not visible on some Linux distros, depending on how much button padding the respective window manager and theme added. Button sizes have therefore been increased in this release to reduce the risk of text not being visible when NetworkMiner is run in Linux.

New Protocol: QUIC

NetworkMiner 3.0 parses initial packets from the QUIC protocol (RFC 9000), which is the UDP based protocol used to transport HTTP/3. The QUIC parser allows NetworkMiner to extract TLS handshakes from UDP 443 traffic, from which the server’s hostname can be read if the client uses the SNI extension. The extracted TLS handshakes from QUIC are also used to generate JA3 and JA4 fingerprints for clients.

New Protocol: CIP and EtherNet/IP

We added parsers for the industrial control system protocols CIP and EtherNet/IP. The implementation does not cover all of the CIP and EtherNet/IP specifications, instead we focused on extracting device information, such as product vendor, product name, bulletin name, serial number and hostname. Such device information is crucial when performing passive asset identification of PLC’s and other industrial devices on OT/ICS networks, such as in factories and power plants. The CIP parser also supports extraction of tag data from Rockwell's proprietary version of CIP.

New Protocol: UMAS

A parser for the industrial control system protocol Modbus/TCP was added to NetworkMiner 2.0 back in 2016. In today’s 3.0 release we’ve enhanced the Modbus implementation to also parse out commands from Schneider Electric's proprietary UMAS protocol, which runs on top of Modbus by using the special function code 90 (0x5a). Our implementation unfortunately doesn’t have full coverage of UMAS, since we don’t have a protocol specification for this proprietary protocol. Nevertheless, our implementation recognizes 40 different UMAS commands (aka UMAS function codes) and can extract fields and parameters from at least 6 of them. The parsed UMAS commands can be viewed in NetworkMiner’s Parameters tab.

New Protocol: REMCOS C2

We started adding parsers for proprietary malicious Command-and-Control (C2) protocols, like StealC, njRAT, BackConnect and RMS, to NetworkMiner a couple of years ago. These malware C2 and backdoor protocol parsers enable security researchers to study what actions threat actors perform when accessing victim computers or honeypot systems.

We’re continuing on our endeavour of creating parsers for malicious protocol by adding support for the Remcos RAT C2 protocol to NetworkMiner 3.0.

Naturally, NetworkMiner’s Remcos parser can’t extract the C2 comms if Remcos uses TLS. Another limitation is that the free version of NetworkMiner is only able to parse Remcos traffic when the C2 server is running on a standard port like TCP 2404. The port-independent-protocol-identification feature in the Professional edition of NetworkMiner, however, identifies and parses Remcos traffic regardless of which port the C2 server listens on (the C2 server in the screenshot above was running on TCP port 1961).

Improved Protocol Parsers

We have also improved several of NetworkMiner’s existing protocol parsers. NetworkMiner’s parser for the trojan/backdoor njRAT (Bladabindi) protocol has, for example, been extended to reassemble full desktop screenshots from njRAT’s Remote Desktop feature.

NetworkMiner’s parser for Modbus has also been extended to support additional function codes and the NTLMSSP parser (for SMB/SMB2) is now better at extracting hostnames to NetworkMiner’s Hosts tab.

Bugs Fixes

A bug in NetworkMiner’s timestamp comparison code previously caused items to be sorted incorrectly when the Timestamp column header was clicked. This bug has now been fixed. We have also fixed a bug relating to extraction of parameters sent in JSON encoded HTTP POST requests.

Breaking Changes

Some of the changes introduced in the 3.0 release might require some users to adapt their workflow. One such change is that the default output path for extracted files and captured packets has changed from NetworkMiner’s directory to %LocalAppData%\NetworkMiner\ in Windows and ~/.local/share/NetworkMiner/ in Linux. This means that you no longer need to add write permissions to the NetworkMiner application directory or subdirectories thereof, since NetworkMiner no longer creates or modifies files there.

Another breaking change is that we have removed the Anomalies tab from the user interface. Windows users can still see alerts by starting NetworkMiner with --filelog, while Linux can use --debug to print debug, warning and error messages to stderr. Use --loglevel warning to suppress info and debug messages.

A change that only affects users of NetworkMiner Professional is that the command line tool NetworkMinerCLI now requires a Corporate License. If you currently have a single-user license, then you will still be able to use the command line tool in your 2.x version of NetworkMiner Professional, but not in the new 3.0 release.

NetworkMiner Professional

There are several improvements in the 3.0 release that only affect users of NetworkMiner Professional. One noteworthy update is that the Pro release has become significantly faster, especially for capture files containing many short TCP sessions. NetworkMiner Professional now saves around two milliseconds in parsing time for every TCP session. This might not sound as much, but it actually makes a huge difference when parsing capture files containing thousands of small TCP sessions.

NetworkMiner’s support for the TLS fingerprinting method JA4 has also been extended even further in the 3.0 release. NetworkMiner Professional now leverages FoxIO’s JA4 database to identify operating systems as well as applications based on client TLS handshake packets.

Other improvement of NetworkMiner Professional include:

• Network operator and AS number displayed on Hosts tab.
• File OSINT lookup includes Censys body_hash lookups.
• IP and domain OSINT lookups added to NetworkMiner’s DNS tab.
• PcapNG packet comments displayed in the Parameters tab.

Upgrading to Version 3.0

Users who have purchased NetworkMiner Professional can download version 3.0 from our customer portal, or use the “Check for Updates” feature from NetworkMiner's Help menu. Those who instead prefer to use the free and open source version can grab the latest release of NetworkMiner from the official NetworkMiner page.

Posted by Erik Hjelmvik on Friday, 04 April 2025 10:53:00 (UTC/GMT)

Tags: #NetworkMiner​ #JA3​ #JA4​ #njRAT​

How to set PCAP as default save file format in Wireshark

Did you know that there is a setting in Wireshark for changing the default save file format from pcapng to pcap?

In Wireshark, click Edit, Preferences. Then select Advanced and look for the capture.pcap_ng setting. Change the value to FALSE if you want Wireshark to save packets in the pcap file format. You have to double-click the “TRUE” text to change it into “FALSE”.

This setting can also be accessed from the Capture tab in Preferences.

I recently learned about this setting from Sake Blok when he commented on my feature request to have Wireshark use pcap as default savefile format instead of pcapng. I have a feeling that this feature request will not be accepted though, since it has received several downvotes. That’s why I’m trying to spread the word about this setting instead, so that everyone who prefers the pcap file format over pcapng can change the default behavior in their own Wireshark installation.

This setting doesn’t affect command line tools, like dumpcap, tshark, mergecap etc. So if you want to capture packets with dumpcap to a pcap file then you need to use the -P switch like this:

Other command line tools in the Wireshark suite, like tshark and mergecap, require that you instead specify -F pcap like this:

What’s Wrong with PCAP-NG?

Why all this fuss about using PCAP instead of PCAP-NG? Well, it turns out that most Wireshark users are happily unaware of just how much metadata there is in the pcapng files they share online. This metadata typically contains information about the CPU of their computer, the exact version and build of their operating system as well as the name of the network interface on which the capture was performed. For Windows users the network interface details even contain a GUID that usually is a world-unique identifier.

I was once even able to identify a person, who had anonymously shared a pcapng file online, by inspecting metadata in the shared capture file github.pcapng. Here's the metadata in that capture file:

This screenshot shows the output from the “Show Metadata” functionality in NetworkMiner Professional. There's also a great way to show pcapng metadata in Wireshark: Open the pcapng file, click View, Reload as File Format/Capture (Ctrl+Shift+F).

Mergecap

The previously mentioned command line tool mergecap, which joins multiple capture files into one, outputs pcapng files by default. In fact, if it is tasked to merge two pcap files (having no metadata), it then creates a pcapng file containing the packets from the two input pcap files enriched with metadata about the computer running mergecap. This metadata is typically information about the operating system as well as the version of mergecap that was used.

Providing an output file with the “.pcap” suffix to mergecap will not help, mergecap still generates a pcapng file. You have to use the -F pcap switch to have it generate a pcap file without metadata about your operating system.

What do Wireshark Users Want?

I recently conducted two unscientific polls, where I asked which savefile format Wireshark should use as default.

In total the polls got 86 votes, where 51 voted for pcap and 35 preferred pcapng. I don't want to draw any real conclusions from these results though, primarily due to the low number of participants but also because there might be a bias among the people who were reached by these polls.

Looking Ahead

I reach out to people I know every now and then when I notice that they are sharing pcapng files containing potentially sensitive metadata. They then have to decide if they are okay with this or if they want to go through the process of replacing the pcapng files with pcap files. In many cases they choose the latter, which can be quite tricky if that involves removing files from GitHub.

I eventually got tired of doing this, especially when I realized that even very skilled Wireshark users often don’t know that pcapng files store metadata about their computers. Reminding people to select the “pcap” format every time they save a capture file doesn’t seem to be the solution. I therefore hope that this blog post can help Wireshark users avoid accidentally sharing unnecessary metadata in the future.

For more information about the pcapng format, please visit pcapng.com.

Posted by Erik Hjelmvik on Tuesday, 25 February 2025 10:33:00 (UTC/GMT)

Tags: #wireshark​ #PCAP​ #pcap-ng​ #dumpcap​ #ASCII-art​

PolarProxy 1.0.1 Released

The new release of PolarProxy generates JA4 fingerprints and enables ruleset to match on specific decryption errors, for example to enable fail-open in case the TLS traffic cannot be decrypted and inspected.

JA4 Fingerprints

JA4 fingerprints provide several improvements over its JA3 predecessor. One advantage is that JA4 fingerprints have a human readable segment that allow humans (as well as computers) to instantly see important features in a client handshake, such as the TLS version and whether or not the SNI and ALPN extensions are used. JA4 is also resilient against TLS extension order randomization.

We added support for rule based matching of JA4 fingerprints in the previous release of PolarProxy. Such a JA4 rule can be used to have PolarProxy take different actions (block, intercept, bypass etc.) based on the JA4 fingerprint of the client’s TLS handshake.

This release additionally includes JA4 fingerprints in the flow metadata that PolarProxy writes to disk when the -f <file> argument is provided.

Flexible Handling of TLS Auth Failures

PolarProxy’s firewall rules now support using TLS authentication error codes as triggers. As an example, the ruleset fail-open.json attempts to inspect (decrypt and re-encrypt) all TLS traffic, except when the client has rejected the server’s certificate at least once during the past 60 seconds. More specifically, it only bypasses decryption if the reason for the rejection was either “bad certificate” or “unknown CA”.

The specific error codes (here 0x0A000412 for “bad certificate” and 0x0A000418 for “unknown CA”) might differ between deployments, since they depend on the underlying TLS library of the PolarProxy machine. The specific values in this example are from a Linux deployment with OpenSSL 3.0.13 installed. Look for the “decrypt_fail_errorcode” messages that PolarProxy prints to stderr to find out what error codes your system is using. You can also run PolarProxy with -v (verbose) or -d (debug) to get even more information about the error codes.

Ruleset Reload on SIGHUP

A PolarProxy ruleset can now be updated on the fly without having to restart PolarProxy. Simply send a SIGHUP signal to PolarProxy, for example pkill -HUP PolarProxy, to have it reload the updated ruleset without affecting sessions that PolarProxy is currently proxying.

If PolarProxy is running as a systemd service, then adding

ExecReload=/bin/kill -HUP $MAINPID

.NET 8

The .NET version has been bumped from 6 to 8 in the 1.0.1 release, which provides better performance as well as long-term support. We've also bumped the System.Security.Cryptography.Xml library from version 4.5 to 9.0.

Posted by Erik Hjelmvik on Friday, 07 February 2025 10:10:00 (UTC/GMT)

Tags: #PolarProxy​ #JA4​ #fail-open​

Blocking Malicious sites with a TLS Firewall

Over 90 percent of all web traffic is encrypted nowadays, which is great of course. However, as HTTP and DNS traffic gets encrypted, defenders have a more difficult time blocking malicious network traffic. One solution to this problem is to use a TLS firewall, which effectively blocks encrypted connections to known bad websites.

DNS Firewalls and Sinkholes

DNS firewalls and DNS sinkholes, like pihole and RPZ firewalls, are simple yet effective solutions that can prevent users from connecting to malicious websites. They work by acting as recursive name servers that deny clients from resolving known-bad domain names. However, more and more DNS traffic is becoming encrypted with DNS-over-TLS (DoT) and DNS-over-HTTPS (DoH), where clients send DNS queries inside an end-to-end encrypted connection directly to a DNS provider. This prevents many DNS based security solutions, like DNS firewalls, from inspecting the queried hostnames.

One way around this problem is to block the actual connections to known-bad domains instead of preventing clients from resolving them. For outgoing TLS connections, such as HTTPS, this can be done with a TLS Firewall.

TLS Firewalls

A TLS firewall inspects client TLS handshakes and extracts the requested server name from the Server Name Indication (SNI) extension. This hostname is generally sent unencrypted in HTTPS traffic (even if you use TLS 1.3), which allows the hostname to be inspected without having to break the TLS encryption. The TLS firewall then checks if the hostname is a known bad or malicious website, in which case the connection is either closed or the user gets redirected to a warning page.

Blocklists

There are several blocklists with malicious domain names, including commercial services as well as freely available lists from ThretFox, CERT Polska and others. These blocklists are often created for DNS firewalls and sinkholes, but they can also be leveraged by TLS firewalls to identify and block traffic to malicious websites.

PolarProxy

PolarProxy can be used as a TLS firewall simply by loading a ruleset that blocks connections to malicious domains.

PolarProxy has the capability to decrypt and inspect what’s inside the TLS encryption, but this feature is not needed when acting as a TLS firewall. The hostname the client wants to connect to is generally provided in the SNI without encryption, so PolarProxy doesn’t have to use the “inspect” action when acting as a TLS firewall. When running in “firewall mode” PolarProxy performs the “block” action for connections to known malicious domains and the “bypass” action for all other TLS traffic. Because of this there is no need for configuring clients to trust PolarProxy’s root certificate in TLS firewall deployments, unless you add a custom rule that decrypts and inspects certain traffic. In fact, if PolarProxy is deployed as a transparent forward proxy in this TLS firewall mode, then zero client configuration is required. This means that managed as well as unmanaged devices, including BYOD, embedded devices, appliances etc., will be protected!

Transparent TLS Firewall (Linux)

If your network has a Linux based firewall that uses iptables, then you’ll be able to run PolarProxy as a transparent TLS firewall directly on your Linux firewall with this command:

You then need to configure the iptables firewall to redirect HTTPS traffic from your network to PolarProxy (see "Routing Option #1" in the PolarProxy documentation for more details).

• sudo iptables -I INPUT -i eth1 -p tcp --dport 10443 -m state --state NEW -j ACCEPT
• sudo iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j REDIRECT --to 10443

Congratulations, your firewall now blocks outgoing HTTPS connections from local clients to known malicious websites!

PolarProxy can also be run in a container using Docker or Podman.

HTTPS Proxy TLS Firewall (Windows)

It’s even possible to run PolarProxy directly on a Windows PC and configure the local proxy settings to send outgoing traffic through PolarProxy. Use the following command to start PolarProxy as a HTTP CONNECT proxy server on port 8080 with a TLS firewall ruleset:

Then configure the Windows PC to use a proxy server on 127.0.0.1 on port 8080.

Add the following exceptions to the Windows proxy settings to ensure that PolarProxy can download the ruleset and blocklists:

raw.githubusercontent.com;*.abuse.ch;hole.cert.pl;zonefiles.io;github.com

Click “Save”.

One side effect of running PolarProxy as an HTTP connect proxy (with --httpconnect) is that this mode only allows TLS encrypted traffic to pass through the proxy. This means that plaintext HTTP traffic that Windows forwards to PolarProxy on port 8080 will be blocked. You’ll see error messages like “Request method "GET" is not supported by HTTP CONNECT proxy” in PolarProxy’s output if it is started with the “-v” argument.

A workaround for this side effect is to run inetcpl.cpl (Window’s old school Internet Properties), select “Connections” tab and click the “LAN settings” button.

Then click the “Advanced” button in the Proxy server section of the LAN Settings window to configure which protocols that should run through the proxy.

Uncheck “Use the same proxy server for all protocols” and remove the proxy settings for everything except “Secure”, which is HTTPS traffic and clock “OK”.

The Windows PC should now only forward HTTPS traffic to PolarProxy’s TLS firewall.

Finally, I’d like to point out that the Windows proxy settings only affect outgoing traffic from applications that respect the proxy settings configured on the operating system. Pretty much every legitimate application will respect these settings and connect through PolarProxy, but there is no guarantee that malware will. This is why a transparent proxy deployment is recommended, such as the one described for the Linux deployment using iptables.

For more information about using PolarProxy as a TLS Firewall and the ruleset JSON format, please visit our TLS Firewall page.

Posted by Erik Hjelmvik on Monday, 27 January 2025 10:45:00 (UTC/GMT)

Tags: #PolarProxy​ #ThreatFox​ #ASCII-art​

VoIP tab in NetworkMiner Professional

The VoIP tab is a unique feature only available in NetworkMiner Professional. The analyzed PcapNG file comes from a blog post by Johannes Weber titled VoIP Captures.

See our NetworkMiner Professional tutorial videos for more tips and hints.

Posted by Erik Hjelmvik on Friday, 04 October 2024 06:20:00 (UTC/GMT)

Tags: #NetworkMiner Professional​ #Video​ #Tutorial​ #VoIP​

Browsers tab in NetworkMiner Professional

The Browsers tab is a unique feature only available in NetworkMiner Professional. The PCAP files analyzed in this video are pwned-se_150312_outgoing.pcap and pwned-se_150312_incoming.pcap, which are snippets of the 4.4 GB Hands-on Network Forensics dataset from FIRST 2015 (slides).

More information about NetworkMiner Professional's Browsers tab can be found in our blog post Analyzing Web Browsing Activity.

See our NetworkMiner Professional tutorial videos for additional tips and hints.

Posted by Erik Hjelmvik on Thursday, 03 October 2024 09:10:00 (UTC/GMT)

Tags: #NetworkMiner Professional​ #Video​ #Tutorial​

Files tab in NetworkMiner Professional

The PCAP file analyzed in this video is pwned-se_150312_outgoing.pcap, which is a snippet of the 4.4 GB Hands-on Network Forensics dataset from FIRST 2015 (slides).

See our NetworkMiner Professional tutorial videos for more tips and hints.

Posted by Erik Hjelmvik on Wednesday, 02 October 2024 07:10:00 (UTC/GMT)

Tags: #NetworkMiner Professional​ #Video​ #Tutorial​

Hosts tab in NetworkMiner Professional

The PCAP file analyzed in this video is MD_2015-07-22_112601.pcap, which is a snippet of the training data used in our network forensics classes from 2015 to 2019.

Techniques, tools and databases mentioned in the tutorial:

• CIDR notation
• Satori
• p0f
• mac-ages

Check out our Passive OS Fingerprinting blog post for more details on how to identify operating systems using TCP/IP headers and browser user-agents.

See our NetworkMiner Professional tutorial videos for more tips and hints.

Posted by Erik Hjelmvik on Tuesday, 01 October 2024 08:25:00 (UTC/GMT)

Tags: #NetworkMiner Professional​ #Video​ #Tutorial​