As more of the internet adopts HTTPS everywhere to secure communications, enterprises rely on inspection tools to examine encrypted traffic to make sure it doesn't contain malicious activity. Unfortunately, the devices intended to verify the security of networking communications appear to be undermining HTTPS, US-CERT warned.
"All systems behind a HTTPS interception product are potentially affected," the Department of Homeland Security's United States Computer Emergency Response Team wrote in its advisory.
The advisory refers to interception products, including inline network appliances like firewalls, secure web gateways, and data-loss-prevention products; client-side software like antivirus; and cloud-based inspection services. Networking and security vendors like Blue Coat, Barracuda, Cisco, Microsoft, Sophos, Arbor Networks, Check Point, Symantec, F5 Networks, Fortinet, IBM Security, Juniper, Trustwave, and Trend Micro include TLS/SSL inspection in their products.
While US-CERT didn't outright tell organizations to stop using these inspection products, it did advise them to ensure that the products they've deployed are performing correct TLS certificate validation. Enterprises shouldn't assume that everything works as expected simply because the products are from recognizable brands. That doesn't appear to be the case for several popular products.
These interception products sit between clients and servers and intercept all encrypted traffic going in and out of the network, decrypt the traffic, inspect the contents, re-encrypt the traffic, and forward the stream to the intended destination. It's basically an authorized man-in-the-middle attack, but it's necessary for enterprises because it lets administrators see what may be hiding within legitimate traffic. Online attackers are increasingly encrypting their activities, whether it's malware communicating with command-and-control servers, crimeware kits downloaded to the compromised endpoint, or files transferred out of the network, and defenders need a way to see and block them.
TLS and the older SSL rely on digital certificates issued by a trusted party to encrypt all communications between a client and server and to verify the server was the client's intended destination. If something is wrong with the certificate, the browser is supposed to display warnings to the user. CERT's warning is based on the fact that in networks where interception products have been deployed, the client is no longer taking directly to the target server.
The browser can see that the connection from the client to that interception product is legitimate, but it can't tell if the rest of the connection is secure or has been compromised. There's no way for the browser on the client side of this equation to see how the product is validating certificates, what ciphers it uses to connect to the server, or whether an attacker has come between the product and the server.
"Because the HTTPS inspection product manages the protocols, ciphers, and certificate chain, the product must perform the necessary HTTPS validations," the advisory said. "Failure to perform proper validation or adequately convey the validation status increases the probability that the client will fall victim to MITM [man in the middle] attacks by malicious third parties."
Popular products fail at security
CERT cited an academic research paper written by researchers at Google, Mozilla, Cloudflare, the University of Michigan, the University of Illinois, the UC Berkeley, and the International Computer Science Institute as the basis of its alert. Titled "The Security Impact of HTTPS Interception," the paper found that network monitoring and security products that can inspect HTTPS traffic often degrade secure communications between clients and servers.
Researchers tested a range of the most common inspection tools and found the majority of them "drastically reduce" the security of TLS connections. The figures are eye-popping: 97 percent of Firefox, 32 percent of e-commerce, and 54 percent of Cloudflare connections that were intercepted by these tools became less secure. Proxies increased connection security for older clients, but the improvements "were modest compared to the vulnerabilities introduced," the researchers said.
An even more damning indictment of network appliances: "A large number of these severely broken connections were due to network-based middleboxes rather than client-site security software."
Of the 12 appliances tested, only the Blue Coat ProxySG 6642 achieved an A rating. Five -- A10 vThunder SSL Insight, Checkpoint Threat Prevention, Cisco IronPort Web Security, Microsoft Threat Management Gateway, and WebTitan Gateway -- introduced "severe vulnerabilities that would enable future interception by a man-in-the-middle attacker" and were given F ratings. Appliances from A10 and Cisco advertised export ciphers, Checkpoint allowed expired certificates, and Microsoft and WebTitan had broken certificate validation.
Barracuda 610Vx Web Filter, Forcepoint Triton AP-Web Cloud, Fortinet FortiGate 5.4.0, Juniper SRX Forward SSL Proxy, Sophos SSL Inspection, and Untangle NG Firewall got C grades. Barracuda and Forecepoint appliances were vulnerable to the Logjam attack, the others advertised RC4 ciphers.
The default configurations for all the appliances tested, other than Blue Coat, weakened connection security, the researchers found. Both the installation process and configuration are difficult on these appliances, and the poor usability is likely the reason why there were so many "abysmal configurations" in real-world networks, the researchers said.
Several manufacturers told the researchers that "secure product configuration was a customer responsibility and that they would not be updating their default configuration." Contrast that to A10's response, which introduced a configuration wizard recommending a "more sane set of cipher suites" last May.
Ten of the appliances supported vulnerable RC4-based ciphers, and five didn't support modern ciphers. This means the client may initiate the connection using a strong cipher, but the appliance would downgrade the connection to a weaker one to finish the rest of the path to the server. Several of the manufacturers told researchers they have deployed updates, and others indicated plans to deprecate RC4 and support modern cipher suites. For example, Fortinet patched the Logjam vulnerability in version 5.4.1, which was released in September 2016.
Administrators using any of the HTTPS inspection products tested in this paper should check version numbers since it's possible the problems have been addressed since the original testing period. If updates are available, they should be applied.
Will Dormann, a senior vulnerability analyst at CERT, echoed the researchers' warnings that inspection products frequently make poor security decisions, such as improperly verifying the server's certificate chain before re-encrypting and forwarding traffic, so clients don't know if they connected to the legitimate server. Some products don't forward the results of the certificate-chain verification, so everyone thinks everything went smoothly even if there were issues with that session. Another common mistake was completing the connection to the target server before displaying the warnings, at which point an attacker can still modify or view the information.
"Organizations using an HTTPS inspection product should verify that their product properly validates certificate chains and passes any warnings or errors to the client," Dormann wrote.
Time to test and verify
There is tendency within the security world to react to warnings in an all-or-nothing fashion. The fact that there are concerns about inspection tools doesn't mean enterprises should stop HTTPS inspection altogether or that visibility over encrypted traffic is bad. Administrators need to be able to see what's happening when an employee uses the internet and when an endpoint has been infected with malware.
Zscaler's Deepen Desai describes how attackers are increasingly hiding their activities within encrypted traffic in the below video, making this kind of inspection important.
TLS/SSL inspection also lets administrators examine application, cross-network, cross-cloud, cross-datacenter and IoT communications for threats. If these communications aren't being inspected, then all the other security defenses in place become less effective.
"Recent discussions about the potential vulnerabilities connected with looking inside of encrypted SSL/TLS traffic ignore the critically important role of SSL inspection," said Kevin Bocek, chief security strategist at Venafi, a certificate and key management company. "SSL inspection is the only way to protect against threats hiding in incoming and cross-network encrypted traffic."
Even CERT is not saying enterprises should rip these products out of the network. Instead, the recommendation is to use badssl.com to verify whether the HTTPS inspection products are properly verifying certificate chains. If any of the tests on this site prevent a client with direct internet access from connecting because of deprecated protocol versions or weak ciphers, then those same clients should also refuse connection when behind an HTTPS inspection product.
"At the very least, system administrators could contact the vendors of SSL inspection software to have them confirm the proper configuration options and behaviors," wrote Dormann.
Administrators can also use CERT Tapioca, a network-layer MITM proxy virtual machine that can check for apps that fail to validate certificates. Based on UbuFuzz, Tapioca is preloaded with the mitmproxy tool to investigate traffic. CERT also recommended taking other steps to secure end-to-end communications, such as upgrading to TLS 1.1 or higher, disabling SSL v1/2/3 and TLS 1.0, utilizing certificate pinning, and implementing DNS-based Authentication of Named Entities.
The CERT advisory has a list of 58 applications "that may be affected by a number of the above-outlined vulnerabilities," but noted they have not been tested, and their presence on the list does not mean they are degrading HTTPS connections. An Arbor Networks spokesperson said Arbor Networks APS isn’t susceptible to this issue because of the way the appliance inspects the traffic.
Administrators should perform their own tests or contact vendors.