Although there have been several good books on the computational aspects and applications of visual languages, this is the first devoted to the theory of visual languages. The aims of visual language research are to understand the classification and specification of visual languages, and to make judgments on their merits. The goals of this book are to provide an introduction to the field; consolidate recent research in visual language formalization; and provide bridges between formalization and other aspects of visual language theory.
The book includes an introduction, an index, and 12 chapters written by leading practitioners in the field who attended the first Workshop on the Theory of Visual Languages, held in September, 1996, in Gubbio, Italy. The 26 contributors represent six countries: 11 are from Italy, the host country; six from the Netherlands; four from the US; three from Germany; and one each from the UK and Australia. Each chapter is a self-contained paper, including references, which total 544. This important and useful book should be in the library of every computer science department, and in the personal libraries of most CS students.
The editors define a visual language as “some set of diagrams which are valid ‘sentences’ in that language. Such a diagram is a collection of ‘symbols’ in a two or three-dimensional space. Which sentences are valid depends on spatial relationships between the symbols. The meaning of a sentence is, in general, constituted by the graphical symbols used in the sentence and by their spatial arrangement” (p. 2). Costagliola et al. provide a simpler definition in chapter 6: “A visual language is conceived as a collection of pictures obtained by arranging graphical objects in two or more dimensions (p. 174).”
Some applications of visual languages are handwriting and sketch recognition; image processing; GUIs; interpretation of user input; design support, including CAD; database interfaces; and spreadsheets. Problems already encountered in developing these applications are a lack of standardization; a high degree of fragmentation in the literature, so that related work is being carried out in various guises, not identified as theoretical visual language research; and a lack of attention to engineering problems by theorists. The editors point out that “visual languages have become a key component of human-computer interaction and seem certain to gain even more importance in the future.” Note, for example, the graphics on the World Wide Web.
Chapter 2 describes some contributions from computer science, and chapter 3 discusses the relationship between visual languages and cognitive science. Part 2 contains five chapters on grammatical formalisms. Chapter 4 covers copy-restricted constraint multiset grammars, chapter 5 is devoted to positional grammars, and chapter 6 presents relational grammars. Chapter 7 is on “Using Graph Grammars to Represent Visual Programs.” Chapter 8 discusses visual data representation. Chapter 9 discusses logical formalisms using Prolog for AI applications. Next, two chapters consider algebraic approaches to visual language theory. Chapter 10 analyzes the relationship between a visual representation and the structure it represents, while chapter 11 presents a syntax-directed approach to visual semantics. The last two chapters discuss the relationship between the meaning of visual expressions to a computational process and their meaning to a human. Chapter 12 discusses “Specifying Input and Output of Visual Languages,” and chapter 13 presents an alternative theory of visual languages motivated by human-computer interaction.
A computer scientist familiar with hierarchies of language generators and parsers will feel quite at home in the first 11 chapters, where such formalisms as context-free grammars, algebraic structures, and logical systems are extended or modified to include visual symbols, such as line segments, circles, and rectangles. Here spatial relations--including disjoint, touching, intersecting, containing, and covering--extend the notion of concatenation for string languages. The examples in these chapters are kept simple to promote readability and understanding and include circuit and transition diagrams. However, the Pictorial Janus example chosen for chapter 9 employs a complex use of Petri nets, is implemented in the Common Lisp Object System (CLOS), and was chosen deliberately to convince the reader that visual language programming is highly complex.
The last two chapters are novel; they address problems of user interfaces. Chapter 12 considers pretty-printing of visual terms and input and output during execution (interactive I/O), using algebraically specified visual languages. The authors of chapter 13 claim that “interactive and end-user computing are the present challenge of computer science.” They propose a definition of visual languages that binds their interactive, communicative, and computational aspects. The general goal is the development of standardized visual languages where users cannot cause unexpected events or become confused by a system’s behavior. This paper provides a formal basis for “assessing the correctness of communication and the control of interaction in visual interactive systems.”
The book fulfills the editors’ three aims admirably. It provides a good starting point for any practitioner or student interested in visual languages, and is well documented, providing direction for further research. The chapters are more explanations, surveys, and descriptions of research areas than reports of research results. Chapters 6 and 12, however, do contain appendices with code for specific examples. The book holds together well as an introduction to and survey of the field.
