Big data will fix internet security … eventually

I’ve always thought that improved computer security controls would “fix” the internet and stop persistent criminality — turns out it might be big data analytics instead.

I’ve long written that only a large-scale improvement of the internet’s authentication mechanisms (that is, pervasive identity) could significantly reduce crime. If everyone on the internet had a default, assured identity, attackers would have a much harder time committing and getting away with cybercrimes.

We’ve seen some progress over the years, such as two-factor authentication and better access controls. The days are numbered for simple logon names and passwords. And though it takes time for defensive controls, warrants, and legal evidence to be collected, efforts on the part of law enforcement are resulting in a greater number of successful prosecutions.

Still, I’m disappointed that pervasive anonymity and weak authentication remain the norm. At the moment, internet crime seems to be at its zenith — and much of society has accepted today’s sad state of affairs as inescapable. They think we can’t do any better.

Nothing could be further from the truth. As the internet matures, legitimate uses will prevail and criminality will shrink. You can bet the bank — or your bitcoins — on that. What I failed to anticipate in the past, however, is the huge role big data analytics would play in securing the internet, our corporate networks, and our personal devices. Big data security analytics might actually account for a bigger piece of the solution than stronger authentication.

The truth is, we’ve had big data security analytics for a while. For example, today’s antispam mechanisms work pretty well. Spam may still account for more than 50 percent of every email sent across the Internet, but very little of it reaches your inbox. Five to 10 years ago, most of what you saw in your inbox was spam.

Then vendors created not only better local email filters, but also began recognizing email patterns early to prevent spam from being delivered. An antispam solution might see the same email sent to hundreds of people or the same IP address issuing dozens of different emails very rapidly, triggering a filter.

Spammers responded by commandeering innocent people’s computers as spam relays and endeavoring to make every spam email unique — but big data analytics can see the hidden pattern.

Another long-used analytic technique is antimalware heuristics. As viruses and other malware used sophisticated permutation engines to appear unique for each user, antimalware vendors started looking for bad behavior patterns during their regular scans. An unknown program exhibiting malware behavior (infecting other files, hiding during boot-up, and so on) gets ranked for each noticed behavior. After enough potentially malicious behaviors accrue, the antimalware vendor marks the program as malicious and assigns it a generic malware ID that most closely matches the behavior.

The top security software vendors are trying to crack the code of accurate, trustworthy computer security analytics. We’re collecting most of the data we need, but we must figure out what gives us the most accurate results — and what data we’re missing. Our early attempts at big data security analytics include companies and services that do the following:

• Monitor command-and-control centers for malicious bots and tell you when your computers connect to those sites, indicating compromise
• Monitor legitimate-appearing network traffic to flag malicious, tunneled traffic
• Track multiple advanced persistent threat gangs and their activities
• Distinguish between legitimate logins and malicious pass-the-hash attacks
• Detect phishing, fraud, and websites using malicious JavaScript redirection
• Tell whether or not a transaction using your identity or financial information is legitimate
• Identify insider data misuse

We’re definitely in the early phases of big data computer security analytics, as this CSO article explains. But the foundation of future security analytics is being laid today.

For a long time we humans have been able to quickly spot signs of compromise. It’s time to let the computers take over some of that task. We still need stronger basic security controls, but it’s clear that big data security analytics will become an ever larger piece of the security puzzle.