This book presents the pioneering work of the late C.S. Wallace on minimum message length (MML), an innovative information-theoretic approach for statistical and inductive inference with a history of more than 35 years.

The entire theory is built on the fundamental idea that “the best explanation of the facts (data) is the shortest.” The general scope of the method is to find the model that generates the shortest description of the data. From information theory, it is known that an event with a probability assigned to it can be represented by a binary message of a length equal to minus the logarithm of the probability. Therefore, the most probable model will have the shortest message. Since any data set can be represented by a finite binary string, an explanation of the data can be considered as a binary message encoding the data, comprised of two parts: the first asserting a hypothesis about the source of the data, and the second detailing aspects that cannot be deduced from the asserted hypothesis. MML is a tradeoff between model complexity and fitting accuracy resulting in models that explain data efficiently with the minimum possible complexity.

The MML method has a strong logical and philosophical background, which is discussed throughout the book. In fact, MML is the formal statement of a fourteenth-century principle known as Occam’s Razor, which declares the preference for the simplest explanation that efficiently describes the data, omitting unnecessary entities.

The method addresses a variety of problems raised in model selection, data explanation, inductive reasoning, and artificial intelligence. MML encoding has been applied to various statistical and machine learning techniques, such as clustering and decision trees.

The content of the book is organized into ten chapters. Chapter 1 is devoted to inductive inference. After an introductory presentation of the fundamental ideas behind the theory and the contents of the book, the author presents an informal account of inductive inference as a fundamental process in human civilization. The chapter also outlines the most important probabilistic and statistical notions that will be used in the sequel, such as random variables, basic axioms and theorems of probabilities, distributions, estimation, model selection, and elements of Bayesian theory.

Chapter 2 introduces, in three sections, the key concepts of information theory needed to understand the length of an explanation. The first two sections give the elementary aspects of Shannon’s theory of information related to coding theory, and an introduction to the theory of algorithmic complexity. The third section is the most important since it connects the notions of information and explanation messages with Bayesian statistics.

Chapter 3 is the starting point for the formal development of methods for constructing short explanation messages and obtaining statistical or inductive inferences from data. The result is a formal characterization of estimator functions (called strict minimum message length (SMML) estimators) minimizing the expected length. However, the computation of such estimators is generally very difficult.

The computational difficulties of SMML estimators are treated in chapters 4 and 5, where some approximation methods are developed. These methods also have severe mathematical difficulties and limitations. As the author claims, the finding of a simple mathematical formula for computing the estimators is still an open problem.

Chapter 6 deals, in a more detailed manner, with some of the approximation techniques proposed for the calculation of the explanation length, aiming to improve the approximating formulae. Some interesting applications of MML to linear regression, function approximation, mixture models, and factor analysis are also given.

Chapter 7 contains applications of MML to the inference of some discrete structures, such as regular grammars, classification trees, binary sequences, and learning causal nets.

Chapter 8 deals with the notions of future and past states of a system. Specifically, the author shows that while the future state of a system is predicted by a deductive process based on the partial knowledge of the present, the past is explained by an inductive process related to the MML principle.

Chapter 9 presents the MML principle as a descriptive theory of the development of science, in the sense that science evolves toward concise explanations and statements of theories.

Finally, chapter 10 contains descriptions of related works, and discusses resemblances and differences with MML.

The level of the book is advanced, as the content is highly theoretical and there are certain requirements for reading it, such as a familiarity with probabilities, mathematical statistics, and elementary calculus. Although elementary information theory and complexity are presented in the book, prior knowledge will enhance the study. In conclusion, the ideas presented in the book are exciting, and can stimulate various discussions and research topics from a wide range of scientific areas. It is strongly recommended for researchers in statistics, data mining, artificial intelligence, and the philosophy of science.