Machine learning is behind many of the systems we typically use, both online and offline, and behind even more of the systems we might use in the future. Given their economic importance, they attract attackers who might be interested in interfering with their correct behavior. Unfortunately, machine learning techniques introduce novel and potentially dangerous vulnerabilities that have not been at the forefront of machine learning research. At least this was the case until the advent of secure machine learning, a subfield that will likely increase in importance in the future.

This book describes the problems introduced by secure machine learning, that is, how machine learning mechanisms can be exploited by adversaries. This is a relevant problem that has already been addressed in specific areas, such as recommender systems [1]. In such systems, attackers try to rig recommendations by means of shilling attacks, whereas the designers of those systems introduce countermeasures either to detect attacks or to make their systems more robust under certain classes of attacks.

In a more general machine learning system, attacks can be performed during training (that is, the process of creating a machine learning model) or while the system is in use (that is, when the learned model is making predictions). In the taxonomy presented by the authors, the former are “causative attacks” and manipulate training data, whereas the latter are “exploratory attacks,” for example, denial-of-service (DoS) attacks would fall under this category. In another dimension of their taxonomy, the authors consider the attacker’s goals. According to this criterion, attacks can be categorized as integrity, availability, or privacy attacks. Integrity attacks compromise systems by increasing their false negative rate, that is, exploiting blind spots so that pernicious activities go undetected and rendering systems unsafe. Availability attacks compromise systems by increasing their false positive rate, as in DoS attacks, rendering systems unusable. Privacy attacks comprise a third category, focused on obtaining information from the machine learning model and compromising the privacy of the potentially sensitive training data used to train that model.

This book, written by researchers from Berkeley, Melbourne, and Google, provides a formal framework for assessing the security of machine learning systems (part of which is the aforementioned taxonomy of attacks). They introduce different scenarios in terms of formal games, in the economic sense of the word “game,” and they survey existing work in this area.

The core of their book is a set of loosely related case studies, taken mostly from the authors’ previous publications. Starting with an academic exercise where the security of a simple hypersphere learner is analyzed, a couple of more interesting examples are covered next: availability attacks on Bayesian spam filters, and integrity attacks on principal component analysis (PCA)-based anomaly detectors. Targeted attacks designed to disrupt Bayesian spam filters cause unreasonably high false-positive rates--for example, up to more than 95 percent misclassification of ham messages, and only one percent of the training dataset is properly contaminated--hence rendering them virtually useless for typical users. The authors also propose a potential countermeasure, called reject on negative impact (RONI), although it is computationally costly. In the case of PCA-based anomaly detection of network traffic using volume measurements, attackers might decrease the detection rate of DoS attacks (up to a ten-fold increase in the false-negative rate). Again, the proper countermeasures can make such systems more robust to poisoning attacks.

The attacks mentioned in the previous paragraph tamper with the training process (they are causative attacks). Two additional chapters explore exploratory attacks. The first chapter, focused on training privacy-preserving support vector machines (SVMs), describes how conventional machine learning algorithms can be adapted to take security issues into account. The second chapter, on near-optimal evasion of classifiers, describes a theoretical model for quantifying the difficulty of exploratory attacks against classifiers. All machine learning techniques suffer from potential blind spots that can be exploited by an adversary, and this theoretical framework tries to quantify how difficult is for an adversary to find a potential vulnerability. In particular, the authors restrict their analysis to convex-inducing classifiers (that is, binary classifiers that partition the space into two regions, one of them convex).

Rather than a comprehensive textbook for the uninitiated, this book reads more like a PhD thesis, even with the leaked occasional reference to “this dissertation.” The introductory chapters are quite formal and provide a good survey of existing work, without explaining too much. The later chapters are more or less independent case studies, similar to what you can find published in research journals and conferences, even when many proofs have been moved to separate appendices so as not to disrupt the overall flow of the text. Finally, the last chapter summarizes the main results of each previous chapter and proposes open research problems that might be of interest to both PhD students and researchers in the field.

The economic incentives of attackers will certainly result in an adversarial environment for the designers of machine learning systems in the future. Most likely, it will be an endless rat race between designers and hackers. Therefore, secure machine learning will somehow end at the forefront of machine learning research. In this academic book, readers will find a first approximation of this emerging field, limited in its scope, yet always with interesting ideas to ponder.

More reviews about this item: Amazon