The fundamental frequency (F0) of human speech is the critical factor in creating synthetic speech with natural prosody, the temporal and rhythmic properties of human utterance that make speech sound natural rather than robotic. Mechanical techniques do a fairly good job of synthesizing intelligible speech, by imitating local intonation or tone contours, but don’t sound truly natural because they don’t handle phrase and sentence fundamental tone contours. Telephone robots have become much more sophisticated, but have not become more natural sounding.

Li, Lee, and Qian take on an interesting challenge, by developing a text-to-speech system for Cantonese, a Chinese dialect with many tones. Intonation in Indo-European languages is employed to convey emotion; however, in monosyllabic agglutinative languages like the Chinese dialects, it conveys lexical, and, to some extent, syntactic information. It seems that the authors’ technique will work for any language if it works for Cantonese, which has nine tones in all, of which three are entering tones, and the other six occur throughout an utterance. Fukienese, with its 13 tones, might be an interesting stress test as well.

The authors capture the change in F0 over an utterance as a phrase curve, and local (syllabic) intonation is detected by any break that exceeds a given length. While the beginning and end frequencies of each of the six tones may vary over the phrase, their ratio or interval will be constant, and the phrase curve will be determined by linear regression over the converted tone heights. The authors analyzed 1,200 utterances, having 4,937 intonation phrases, to develop a Cantonese text-to-speech system, called CUTalk, consisting of three modules: text analysis, acoustic synthesis, and prosody generation.

Subjective tests of the system were made with sentences taken from local newspapers, and naturalness was rated, on a scale from one to five, by native speakers. The results showed a marked improvement in the generation of natural spoken Chinese by a computer, but also revealed some opportunities for additional improvement. This always seems to be the result of trying to analyze or synthesize human language mechanically; we discover that language is even more complex than we thought, and learn as much about the natural language as we learn about computation for linguistic applications.