Deep packet inspection is dead, and here's why

Deep packet inspection is a methodology that network security professionals have been doing for many years. It involves looking at the data going over the network and determining if anything malicious is going on based on what's in those packets.

When I was cutting my teeth on Solaris back in the late 90's, we used snoop [1] to grab a packet capture to troubleshoot issues. It had a really cool flag (-a) that would pipe the packets to /dev/audio so that you could listen to the packets as they crossed the network. I guess this was the closest thing to remind us of the modem handshake we all learned to love [2].

With Linux we got tcpdump [3], which gave us a myriad of extra stuff we could do to filter the packets, and we all got good with hexeditors and the like to pull the packets apart.

Soon after came snort [4], which allows us to alert on signatures we saw in the packets. This was a game changer because now we could start getting alerts if we saw something nefarious. This was one of the first popular Intrusion Detection Systems (IDS) [5] and it performed Deep Packet Inspection. It looked deep inside the packet, past the IP address, port and protocol, and looked at the payload of the packet itself. When a new exploit came out, someone wrote a signature for it and shared it. Everyone who installed it could now detect the exploit going into their systems. Life was good!

Wireshark [6] (nee Ethereal) came out a few years afterward which allowed for better access to protocol analysis, which really helped writing signatures and life was even better!

By this time, Banks and high security websites started encrypting their data because these tools could also "sniff" out credit card numbers and usernames and passwords. There was a hayday for criminal hackers everywhere who could easily pull these things out of the air. So we used our IDS's to see when sensitive data was being sent in the clear and quickly cleaned them up. Chaosreader [7] was a lot of fun!

Then someone said, "Hey, I create firewall rules based on my IDS and that is work. I want to be lazy and have the IDS write the rules for me." And thus was born the Intrusion Prevention System (IPS) [5]. And whole industries were born with these systems being sold like hot cakes. The firewall vendors saw an opportunity and created Next Generation Firewalls (NGFW). These had an IDS built in that talked to the firewall and made it into an IPS.

But now, we are seeing a huge push to encrypting all Internet traffic. 74% [8] of colleges and universities are now encrypting their home pages by default. Social media, news outlets, online stores, etc, have all followed suit as well. The new version of the web protocol, HTTP/2 requires encrypted communications to even work.

We don't even use telnet anymore in preference for the encrypted ssh!

So what does all this encryption mean? It breaks Deep Packet Inspection. All the IDS's, IPS's and NGFW's that we bought are becoming obsolete. They can't inspect the encrypted packets. Of course they try to hold onto this technology by introducing technologies like SSL inspection (aka SSLbump [9]). This technology basically breaks the trust model of Internet encryption by acting as a man-in-the-middle. The place where you work spoofs itself as the encrypted site you are going to. Because they control your computer, you don't even know it is happening. Then they decrypt your Internet traffic to use DPI on it and then re-encrypt it back to the Internet.

This sounds benign, but in reality, it is a huge invasion in privacy. This means that they now have access to your credit card and bank account information, your health records and the funny picture of your dog you just privately shared with your significant other. And they did it without telling you that they were going to do it. As a user, this upsets me.

As a security professional, setting up one of these systems might lead to the erosion of trust with my users. We have been saying for years that the major problem with computer security isn't the computers, it's the lack of knowledge of the human operating the computer. We are trying very hard to train people how to compute safely, which requires them to trust us. When we start looking under the covers at what our users are doing to stick with a paradigm we are used to, we only make things worse. And for those of you who say, "they'll never know," I say, they'll find out. We are seeing vendors like Google pinning certificates in their software to ensure SSL inspection is detected and the user is alerted. Of course, there is probably a valid use case for SSL inspection out there somewhere, but for the majority of systems, I think it is a Bad Idea.

We've also found over the years that signature based security mechanisms aren't as effective as they once were. Criminals have access to the same tools that we do and they test their software against them to make sure they aren't detected. This is why anti-virus companies are using heuristic analysis [10] to detect malware. This is why companies are using network behavior anomaly detection (NBAD) [11] systems to determine unusual events on the network.

Instead of holding onto deep packet inspection, I think we need to transition to new methodologies for detecting bad things on the network. Telemetry data is one of these ways through passive monitoring of netflows [12] or DNS queries [13]. By looking at traffic on your network and determining what looks anomalous, you may be able to determine where the nefarious activity is happening. By looking at your DNS queries and investigating Passive DNS [14] with Bind RPZ [15] or using OpenDNS [16] you can cut down on a huge amount of bad sites on the Internet and interrupt phishing campaigns and malware.

Hope this has been informative and made you think about the future of deep packet inspection!

Thanks,
ep

[1] https://docs.oracle.com/cd/E23823_01/html/816-5166/snoop-1m.html
[2] https://www.dialupsound.com/
[3] https://en.wikipedia.org/wiki/Tcpdump
[4] https://snort.org/
[5] https://en.wikipedia.org/wiki/Intrusion_detection_system
[6] https://www.wireshark.org/
[7] http://chaosreader.sourceforge.net/
[8] https://security.ias.edu/sites/security.ias.edu/files/fun_with_certific… slide 7
[9] http://wiki.squid-cache.org/Features/SslBump
[10] https://en.wikipedia.org/wiki/Heuristic_analysis
[11] https://en.wikipedia.org/wiki/Network_Behavior_Anomaly_Detection
[12] https://en.wikipedia.org/wiki/NetFlow
[13] https://en.wikipedia.org/wiki/Domain_Name_System
[14] https://www.farsightsecurity.com/technical/passive-dns/passive-dns-faq/
[15] https://dnsrpz.info/
[16] https://www.opendns.com/