The State of AI in Cyber Security in 2022

In a recent IBM-backed report around AI in cyber security adoption, 45% of respondents expressed concern over their ability to understand the real value or truth of vendor claims regarding their own AI cyber security products⁵. That's no minor misgiving in a business climate where almost half of all AI-based tech startups are reported to actually use no AI in their products⁶.

Many of the common terms used to describe AI and machine learning (two phrases still at the center of their own semantic dispute) can be misleading. For instance, the term 'algorithm' is often misunderstood in the context of AI. Some of the most famous examples of this conflation are Google's search algorithm, Netflix's recommender system, and Facebook's tailored advertising and timeline-post decision processes—the headline-grabbing code from some of the world's leading researchers into machine intelligence.

All these systems have enormous amounts of proprietary user data to train on, and all of them leverage machine learning—but not directly. Typing Game Of Thrones into a Google or Netflix search box doesn’t immediately create a new data point in a neural network.

You could hardly expect a reasonable response time if it did: even the significant resources of these global players are equaled or exceeded by high-volume queries⁷ from customers expecting almost immediate results across a range of connectivity conditions and device types.

Rather, machine learning networks are used to design more efficient 'traditional' algorithms: static code that can handle and act upon data mechanically and predictably based on designed rules. The difference is that, increasingly, these rules will have been updated, amended, and sometimes even determined by AI. Updates may be very frequent, even in the order of minutes or less; but these interactions rarely offer a direct interface with a responsive AI or ML.

This does not signify that algorithms are 'impersonating' AI, but rather that AI can provide better algorithm development for the data-heavy and high-volume network environments where machine learning can't currently be expected to operate. Or at least, to operate with the same response rate or surety of accountability.

This is important in a cyber security context, where latency is often such a critical factor that only pre-optimized algorithms are able to respond to incursions or anomalies quickly enough.

In some cases, which we'll examine later, network connectivity can either be insufficient, not available, or not recommended for an AI-based protection system. In these instances, a 'local' machine intelligence must gather, cogitate, and act within preset parameters.

This 'standalone' AI will likely be dealing with a very targeted or filtered set of data points (such as a facial recognition task), and it will usually be running in a minimalist, even frugal implementation.

Since the advent of mobile and low-powered neural networks makes this scenario increasingly likely in the next 5-10 years⁸, and since 'real-time' Convolutional Neural Networks and other ML frameworks seem set to rise⁹, it's worth understanding the practical trade-offs between a 'live' AI system and an algorithm.

Applied AI insights are best suited to recommendation systems, customer services and predictive maintenance, whereas highly curated algorithms are better set to tasks such as credit risk assessment, insurance underwriting, and claims processing.

These use cases derive from the contrasting strengths of each approach:

Even if practical considerations didn’t make them necessary, algorithms derived from machine learning systems provide a useful layer of curation and human control in a culture still very nervous¹⁰ at the prospect of 'autonomous' AI.

Since the growing speed and responsiveness of live neural networks is likely to remove the need for this layer of interpretation in the next 10-15 years, we'll have some hard decisions to make at that point about the discriminatory powers granted to AI frameworks. For now, it's enough to understand the relationship between the algorithm and the 'live' machine learning system that creates it.

So, with these and other restrictions and limitations, how convinced is the security sector about the current benefits of implementing applied AI techniques?

One of the most compelling reasons for enterprise IT teams to adopt AI is that malefactors are doing likewise¹⁴. The open-source nature of global AI and machine learning research means that powerful tools with criminal applications can be crafted from common code repositories and projects.

In 2018, a panel of experts gathered in the UK to explain the potential ramifications of criminal AI¹⁵. The implications put forward range from persuading mining companies to bid for drilling rights in unsuitable locations to the creation of AI-powered viruses capable of performing far more user-specific damage from an infected consumer machine than the customary inconveniences (such as the mailing of spam messages, enlistment in a botnet, or the installation of crypto-mining software).

It's a chilling picture of a new breed of infection that will seem much more 'personal' to the victim, as infected systems are exploited to gather up, collate, and intelligently interpret data in the most damaging way possible.

A 2018 report from the Universities of Oxford and Cambridge (together with the Electronic Frontier Foundation and several other institutes) proposes that governments around the world devote greater resources to assessing their state of preparedness for AI-based nation state and criminal attacks in the context of a locus of global research that is currently accessible to all¹⁶.

It also suggests a new mandate for transparency—aka the 'Blade Runner Law'—to distinguish the actions of machines from human action; and that the community should work with governments to define and divide clearly the responsibilities and liabilities of the state and private sector in relation to the proliferation of AI-based attack vectors and their consequences.

These are matters of policy and governance that are set to affect the cyber security industry in the next five years, irrespective of the action it takes or doesn't take. At the moment, many of the issues raised seem abstract and conjectural.

In reality, they may have a direct bearing on a company's future viability. An enterprise that has not committed to AI-based deployment will be the least prepared to protect its interests in a global research environment where public AI code can be easily weaponized by bad actors.

With these as some of the motivating factors for the adoption of AI in the security arena, what are some of the areas where the innovative use of applied machine intelligence might be most productive?

The phenomena of counter-intelligence and propaganda have extended from political and cultural topics into relatively new sub-sections of cyber security over the last five years. Whether the risk is overstated or not, inaccurate information (lies) is becoming synonymous with malicious information (exploits), from a safety perspective.

This is partly because the damage such techniques can wreak has become so frequent and quantifiable recently, but mainly because of the extent to which they have become subject to programmatic, automated processes. Attack vectors now can be greatly facilitated by machine learning, and potentially repelled by AI information systems

Consequently, 'fake news' is increasingly discussed in the context of weaponized data¹⁷, capable of causing harm comparable to a database leak in terms of reputation and commercial impact. Such attacks are proving viable in both the public and private sectors.

Facebook, one of the global leaders in AI research, has committed¹⁸ to using machine learning techniques in an effort to distinguish fact-based news reporting published in its timelines from terrorist propaganda which uses the same journalistic constructs and conventions of language.

Since the difference can be subtle and semantic, it's a considerable task for Natural Language Processing (NLP) analysis, even for a company like Facebook that has such a high volume of sample data to analyze.

Indeed, the sheer scale of the big data era is probably the defining factor in the move toward AI-driven cyber safety. One company (since incorporated into Twitter) that specializes in identifying fake news found it necessary to actually redefine the 'volume problem' with a system called Geometric Deep Learning.

New models for disinformation have emerged recently. Deepfake video manipulation applications— a new stratum of AI software first revealed in late 2018¹⁹ that uses autoencoders to misrepresent famous personalities in manipulated footage— have become an increasing concern as the 2020 US election nears²⁰.

Previous techniques for identifying AI-generated fake footage have included the blink test (which is relatively easy for fakers to obviate with better curation of the source data) and analysis of lighting and breathing rates. At the time of writing, a new technique has been developed²¹ that uses the correlation between facial expressions and head movements to provide a potential deepfake signature. However, experts acknowledge that deepfake software can adapt quite easily to new 'tells' as they come to light²².

But this model of an 'algorithmic arms race' now applies to a much wider context in AI-centered cyber security.

Fake news is not entirely centered around AI-driven methods—more rudimentary manipulation can be just as effective²³. But counter-measure AI techniques remain applicable for these cases too, in concert with potential new data-led approaches such as video-hashing and blockchain accountability.

The historical foundations of the internet still underpin most of the recent innovations and protocols in the global network infrastructure³⁰. Therefore, network safety is one area where dealing with legacy technology is inevitable, both in terms of the security arena and applied systems.

The stacked nature of this super-scale 'legacy' architecture also has a direct bearing on the security of user authentication techniques and other, more automated network protection measures, such as Comodo SSL. Consequently, sweeping change seems unlikely either in the attack surface or the systems that have grown to defend it.

However, this is not why AI has made so little impact on the problem of network intrusion detection, relative to (apparently) similar sectors such as natural language analysis, spam detection and recommendation systems. Rather, it's because the core problem of anomaly detection is not as similar to AI's most fruitful areas of research as it may appear³¹.

Machine learning systems excel at discovering relationships between disparate data points and at finding similarities that may be difficult, labor-intensive, or uneconomical to identify manually. During training, an ML system might cycle through petabytes of relevant data, where even a low eventuality of target behavior scales up to an operable mass of relevant data.

But the proposition for anomaly detection is not like this: network incursions are historically and statistically rare³², meaning that the system must learn to identify 'outliers' using training sets which are out of date or highly unbalanced—or both.

A mission-critical machine learning system needs a perfect model of normality in order to establish a baseline for the identification of incursions or abnormal events. A model that is both accurate and resilient to the appearance of previously unknown types of abnormality can be difficult to generate or maintain in these circumstances.

The cost of failure, measured in terms of 'ignored' anomalies or false positives, is much higher than in semantically similar machine learning pursuits such as optical character recognition, spam detection or product recommendation algorithms.

Neither is it possible, as in those less critical research sectors, to generate datasets for training purposes that feature authentic events of failure at scale, due to the innovative designs and signatures that we can logically expect from new approaches to network incursion.

AI systems face a similar paucity of data in the field of predictive failure analysis, where the low frequency of 'abnormal' data can require new approaches to the dimensionality of models aimed at the task.

AI-driven network intrusion detection is not the lowest-hanging fruit in the present wave of business interest in machine learning. Current commercial offerings in this line are few, mostly centering around analysis rather than real-time response.