Add PCAP-over-IP support

[mmguero]

Submitted by @erik4711 as cisagov#278

💡 Summary

Add support for PCAP-over-IP (aka PcapOverTcp) to allow Malcolm to read a continuous PCAP stream of network traffic from a remote machine. This feature also enables reading decrypted TLS traffic from a TLS inspection proxy.

Motivation and context

Individual tools that handle PCAP files or network traffic have support for reading PCAP-over-IP. There is for example a Zeek plugin Zeek::PcapOverTcp, which can read PCAP data from a remote sniffer or a TLS decryption proxy. However, it would be helpful if Malcolm would support reading PCAP-over-IP centrally, and then provide that PCAP data to installed tools like Suricata and Zeek.

There is support for pcapReadMethod=pcap-over-ip-client and pcapReadMethod=pcap-over-ip-server in Arkime, which might be of help here.

There are a few less desirable workarounds for reading remote PCAP data into Malcolm, such as using sftp to copy pcap data into Malcolm.

Implementation notes

A desired implementation would be a service that either listens on a TCP port for incoming PCAP-over-IP connections, or a client that actively connects to an IP:PORT to read PCAP-over-IP data. The received pcap/libpcap data should be expected to be compliant with the PCAP file format , not PcapNG.

Acceptance criteria

A simple test case would be to make a PCAP file available to Malcolm via a local netcat listener like this:
nc -l 57012 < sniffed.pcap

If Malcolm can read and import the packets in sniffed.pcap via the netcat listener, then we've succeeded.

A more advanced test would be to have Malcolm read decrypted TLS traffic from PolarProxy, which is started with --pcapoverip 57012 or --pcapoveripconnect <Malcolm-IP>:57012.

[mmguero]

Kamino closed and cloned this issue to cisagov/Malcolm