Based on the disturbing number of successful data breaches over the past few years, it's pretty evident that organizations are being overwhelmed by the growing number of threats.
However, a new breed of security solution has sprung up, offering to apply machine learning to enterprise security. These tools deliver the ability to analyze networks, learn about them, detect anomalies and protect enterprises from threats.
So, is machine learning the answer to today's cybersecurity challenges? Industry analysts and companies offering these products say they're seeing increased demand, and the early reaction from users is positive.
"Machine learning is the major security trend of 2016," says Eric Ogren, senior security analyst at 451 Research. "Every security officer now knows that behavioral analytic products offer the best chance of catching attacks that elude static preventive defenses."
And machine learning is the heart of behavioral approaches, he says. "There is nothing like watching, listening and learning," Ogren adds. "Machine learning observes behavior in defining a statistical profile of normal activity for a user, device or Web site. This is important, as it provides the foundation for behavior analytics to prevent major damage from attacks that slip by anti-threat defenses or abuse authorized activity."
A long-term benefit of machine learning is that it sets an organization on the path toward a probabilistic and predictive security approach that integrates smoothly with generally accepted IT practices, Ogren says. "We are seeing this pay dividends already in major cloud and media enterprises, where security is measured less in the 1s and 0s of good and bad and more in reducing the risk of a major business disruption that can flow right to the bottom line."
Potential challenges
As with any newer technology, machine learning presents potential difficulties. "It can be challenging to differentiate the quality of machine learning algorithms across different vendors," Ogren says. "Quality will come out in the results. We recommend that proof of concept projects focus on a few discrete use cases for users, devices and Web sites to demonstrate product effectiveness."
Although machine learning can lead to huge improvement in security, "it is not the end-all be-all," notes David Monahan, research director, security and risk management at research firm Enterprise Management Associates Inc. "It has its limitations and best applications. It is a great tool for much of security to identify things that are out of the ordinary and should be evaluated or investigated."
There are two main types of machine learning used in security: supervised and unsupervised. "They work better for different things, but in the end they find anomalies in data sets provided," Monahan says. "Therefore, it is only as good as the data provided. So [machine learning] is an additive technology, not a foundational technology."
Key benefits
The key benefits of the technology are its ability to detect trends, patterns and anomalies in large and diverse data sets and the speed at which it can do this, Monahan says.
"It is faster by far than most if not all big data tools, as it can work in real-time to near real-time -- seconds to minutes -- and it does not need to wait for batching data sets." The need for machine learning is driven by two facts, says Kris Lovejoy, president and CEO of BluVector, which provides security technology that uses machine learning.
One is that it takes a long time to detect a compromise, and another is that in many if not most cases companies are informed by a third party that it has been breached.
"Organizations need capabilities that allow them to get in front of the threat, finding and eradicating them before they can do harm," Lovejoy says.
Companies "have realized that they can't anticipate every possible attack vector, and they can't afford to manually create rules that detect the vectors they have anticipated," says Mike Paquette, vice president of products at Prelert, another provider of security tools that use machine learning.
"They're looking for a way to automate the analysis of their security-related log data in such a way that these elementary attack behaviors are detected on a continuous basis," Paquette says.
Here's a brief sampling of available security tools that leverage machine learning:
- Acuity Solutions provides BluVector, a malware detection and cyber hunting product that uses machine learning as the mechanism for identifying and prioritizing potential threats. As these threats are identified, forensic packages are created for hunters and responders tasked with investigating and triaging the threats.
- DgSecure Monitor from Dataguise is a data breach detection product that uses machine learning and behavioral analytics to generate alerts whenever user actions deviate from the typical behavioral profile. Whether sensitive data is protected or not, DgSecure Monitor makes it easy to create data security governance policies using this capability in combination with user-defined policies.
- Deep Instinct offers a product called Deep Learning that's inspired by the brain's ability to learn to identify an object and turn its identification into second nature. By applying deep learning to cybersecurity, Deep Instinct uses this process for two phases: learning and predicting. The result is instinctive cyber protection against even the most evasive cyber-attacks, from any source.
- Distil Networks offers technology that protects Web applications from malicious bots, API abuse and fraud. Each Distil customer benefits from a global machine learning infrastructure that analyzes attack patterns in real time. For example, Distil proactively predicts a bot based on correlating more than 100 dynamic classifications and pinpoints behavioral anomalies specific to a site's unique traffic patterns.
- Prelert offers three advanced threat detection products that use machine learning technology for security. All three are built around Prelert's behavioral analytics engine that uses unsupervised machine learning technology to create baselines of normal behavior in companies' log data, and identify anomalies or unusual patterns in the data that are related to cyber-attack activity.
Bank on it
Companies using machine learning technology report early success. Orrstown Bank, a provider of community banking services, began using machine learning technology to address the rampant growth of credit and debit card fraud.
"Card fraud has been on the rise due to a few reasons, but primarily because of the volume of card data breaches from large and small merchants," says Andrew Linn, senior vice president and CISO at the bank. "Fraud detection solutions either provide only rudimentary detection capabilities or are too expensive for the average community bank."
Orrstown partnered with Prelert to use its machine learning technology to tackle the card fraud problem. Although originally designed to detect anomalies among technology assets, Orrstown has found the that Prelert's product can also detect anomalies in human behavior, including human card usage behavior.
"Fraudsters often follow a purchasing pattern when using stolen cards," Linn says. For example, they make an initial and usually inexpensive purchase to verify that the card is still active and working. If that test transaction goes through, they quickly execute a series of other, higher dollar amount transactions.
The fraud scoring engine based on machine learning from Prelert helps the bank detect the first fraudulent transaction so it can stop the subsequent fraudulent transactions that are higher dollar amounts.
The technology identifies fraud by detecting anomalies in the card usage across multiple dimensions -- time of day, dollar amount, location, type of merchant, etc. -- combined with expert knowledge about patterns of known fraudulent transactions that Orrstown supplies.
"Although we've only recently operationalized this solution, early results indicate that we can reduce our fraud losses by up to 50%," Linn says.
That's the ticket
Another user of machine learning, ticket resale services provider StubHub has been integrating Distil's technology for about 18 months. "As new security threats have surfaced, Distil has become an integral part of StubHub's larger security strategy, especially to combat account take-overs," says Marty Boos, senior director of technical operations at StubHub.
The machine learning capability of Distil's offering learns from the patterns that it detects within the traffic coming to StubHub, so it can begin to predict how bad bots and other security issues will evolve, Boos says.
StubHub and Distil are able to collaborate daily to identify what's happening now and what StubHub expects to happen in the future. "As bots and other types of malicious traffic quickly evolves, networks and platforms have to be diligent about getting ahead of new tactics," Boos says.
At StubHub, a purchase often constitutes an immediate transfer of a digital good, "so it's critical that we keep bots and other threats from compromising our network," Boos says.
"It's a business risk. Distil helps us to be smarter about how we deal with current issues and prepare for the ways the threats of the future will evolve."
Human Longevity Inc., which provides technology for creating the world's largest and most comprehensive database of whole genome, phenotype and clinical data, started using Darktrace's Enterprise Immune System in September 2015 to characterize what it considers normal network activity across its business and corporate platforms.
"The goal was to determine any abnormal activities across our network and have our teams focus on analyzing those anomalies in order to determine their threat level," says Tom Brandl, head of IT security.
"Darktrace's machine learning technology learns the pattern of life in our environment -- gaining an understanding of what is normal for our network so it can then identify any abnormal activity," Brandl says. "This allows our programmers and specialists to do what they do best: examine those anomalies identified by the Enterprise Immune System and determine the level of the threat and the actions to be taken."
The biggest benefit of the technology is that it has given the company much better visibility and understanding of what is happening in its environment. Those in the market say the future will bring dramatic new capabilities in efforts to strengthen information security.
"There is no theoretical limitation to creating an artificial version of the human brain," says Guy Caspi, CEO, Deep Instinct. "Deep learning is bringing us closer to this goal at a great and accelerating pace. We can expect many exciting breakthroughs in the upcoming years, especially in unsupervised learning."
While deep learning has successfully been applied to computer vision, speech and text understanding, "there are many other challenging domains which deep learning can potentially revolutionize," Caspi says.
With machine learning at the center of artificial intelligence and data science, "it will continue to drive innovations in development and learning algorithms," says Venkat Subramanian, CTO at Dataguise.
"The technology is in adoption across all industries where data-intensive analysis is taking place, and the enormous adoption of big data is one trend that is accelerating its integration throughout analytics," Subramanian says.
"This will continue to span all areas of computing and will be especially useful in the detection and defense against violations to corporate security and unwarranted access to sensitive information."
Violino is a freelance writer. He can be reached at bviolino@optonline.net.
This story, "Machine learning offers new hope against cyber attacks" was originally published by Network World.