Next Previous Contents

7. Source Code Documentation

This section is of little use for anyone except for programmers willing to contribute to the development of Epos or going to modify its source code. It is also not trying to become a beginner's guide to Epos. Anyway, if you are personally missing something here or elsewhere, tell me and I may add it; that will become almost the only source of progress in this section of documentation. The section may also slowly become outdated due to lack of interest.

7.1 Design goals

Overall coding priorities, approximately in order of decreasing precedence:

7.2 Isolated classes

class parser, unit, rules, text and maybe a few others are isolated classes that take no advantage from inheritance. The reason for the class-oriented design is just a matter of code readability and decomposition in this case.

Class simpleparser

This class takes some input (such as a plain ASCII or STML text) and then can be used in conjunction with the class unit constructor to build the text structure representation. Its purpose is to identify the Latin text tokens (usually ASCII characters, but some traditional tokens like "..." would be difficult to identify later, as well as numerous other context dependent uses of "."). The parser also identifies the level of description which corresponds to the token and this is the information needed by the class unit constructor to correctly build the TSR. In this process, the parser skips over any empty units, that is, units that contain no phones (simple letters) at all.

Note that it is unnecessary and counterproductive to distinguish between homographic tokens used at the same level of description here; such intelligence can be handled more flexibly by the language dependent rules. In fact, they tend to be usually language dependent. The parser only avoids losing information (through empty unit deletion) by the minimum necessary tokenization.

The STML parser is still unimplemented.

Class unit

This class is the fundamental element of the text structure representation. Its methods are listed in unit.h. Every object of this type represents a single text unit. Every unit includes pointers to its immediate container, and to its contents. The contents are organized in a bidirectional linked list; pointers to the head and tail units of this lists are stored in the unit. These links, i.e. prev and next, also serve to locate the neighboring units; they may be NULL, indicating that this is the first/last unit in the immediate container. For most uses, these pointers are not suitable to be used directly; the Prev and Next methods find the neighbor, even if a higher level boundary lies in between. It is also possible to mark a unit as a scope one. In this case, the Next and Prev methods will be unable to cross its boundary from inside out (they will return NULL if this is attempted). If you need to modify the TSR directly, you will benefit from calling unit::sanity occasionally. This method checks the TSR structure near the unit which has called it and will report a severe error, if an invariant is violated, thus saving you from misguided error messages or crashes later.

To extract the prosodic information from a TSR, call the effective method. It will combine the prosodic adjustments present at all the levels of description above the current unit.

Class text

This class represents a logical line-oriented text file. It handles things like the @include directive, backslash-escaped special characters, initial whitespace and comment stripping. It is used for the rule files and configuration files, but not for the dictionaries.

Class file

This class represents a physical data file. Its main purpose is to cache and share files repeatedly needed by Epos. The claim function (to be found in interf.cc) should be used for opening the file (or only sharing an existing copy if the file is already open) and reading the data out of the file. The unclaim function is called separately for every claim call whenever the file is no more needed.

Any code which uses this class should never extract the data member out of it and use it independently, even if the class itself remains claimed. This is because if the content of the file has changed, the data in memory will be reallocated and re-read upon the next call to claim or possibly even sooner. This may cause invalidation of the original data member at any point of a control switch to another Epos agent. It is possible to call reclaim at any time to force re-reading any file if its time stamp has changed.

Class hash

class hash is derived from class hash_table<char,char>. The hash_table template is a generic hash table, keys and associated data items being its class parameters. This implementation uses balanced (AVL) trees to resolve collisions and is able to adjust (rehash) itself when it gets too full or too sparse. It is a very robust and fast implementation and it is independent of the rest of Epos, so you may use it in other projects if you want to (subject to GPL). If you want to have the hash table keep a copy of its contents, the key and/or data may only be of a fixed size type, or a C-style string. Alternatively, the hash table will only store pointers to these items. These approaches can be mixed in any reasonable sense of "mixing".

The hash tables are used frequently in Epos in various type combinations (see hash.cc for a list. They're also used for parsing the dictionary files.

Class rules

Note the difference between class rules and class rule. Every set of rules in Epos (there is one per language) is a class rules, which contains a single r_block, which in turn contains the individual rules. The class rules serves as the only communication interface between the rule hierarchy and the rest of Epos, but there is no inheritance relation between them.

7.3 Class hierarchies

Class rule

Each rule object represents a rule to be applied to a structure of units. The class hierarchy:

rule

Classes not beginning in r_ can be considered abstract.

Class agent

Epos can be configured to support multiple simultaneous TTSCP connections and except for bugs, no single unauthorized connection should be able to create a Denial of Service situation, such as long delays in processing other connections. To achieve this, Epos uses a simple cooperative multitasking facility called agents. An agent (process) is an entity, which is responsible for carrying out some task, such as reading a few bytes from a file descriptor. At any moment (except for the startup and the very moments of a transfer of control), exactly one agent is active (Epos doesn't support SMP to avoid the unnecessary overhead and complexity in the typical case). If an agent has to wait for some event before its job is finished, for example, when the sound card reaches full buffers or not enough data has arrived through a network connection, the agent calls the block method (reading) or push method (writing) with the offending file descriptor. It is also possible for an agent to wait until some other agent executes; see the block and push methods' implementation for an example. If an agents wants to have another agent running, it can call the schedule method to add it to the queue of runnable processes. The scheduled agents always acquire control through the run method; when this method returns, another agent is chosen. If there are no more runnable agents, Epos will wait until an agent becomes runnable through a status change of the file descriptor the agent is blocking for.

Most agents get their data input through the inb data member and place their output into the outb data member. Whenever the agent has completed a stand-alone chunk of output, the agent calls the pass method to pass it to its successor and to schedule it for processing. The output agent never calls pass (it has actually no successor and it is responsible for writing the data somewhere outside Epos), but it calls finis when the data has been successfully written.

Most agents are organized into streams of interconnected agents. See the strm command for the semantics of that. Other agents are responsible for individual TTSCP connections, for accepting new connections and other tasks. A special agent is used for deleting other agents when they need to delete themselves.

The chunk agent may perform utterance chunking, that is, splitting the text being processed at convenient points, such as after a period or at end of paragraph. Such chunks travel through the rest of the stream independently and they are queued between consecutive agents if necessary. Such a queue (if non-empty) is a linked list starting with pendin of the latter agent while the end is pointed to by pendout of the former agent. The pendcount member of the latter agent stores the current number of data chunks in the queue, which is used for sanity checking and flow control.

The current agents are:

agent

7.4 Testing

The Epos package contains three TTSCP clients. One of them is the standalone say utility, which is provided as a good and simple example of a TTSCP client. We suggest to use it as a starting point for developing specialized TTSCP clients, even though it is already somewhat crufty.

The tcpsyn virtual synthesizer also embeds a TTSCP client; it is wise to check its proper functioning after making changes to the TTSCP server implementation.

A standalone test suite is compiled under the name vrfy. It is currently only trying a few standard tricks to crash the server and is far from being a rigorous test suite. However, it manages to catch much more programming errors than say and we recommend to run it after making any changes to the source code of Epos. This test suite assumes Epos has been configured correctly and is listening at the standard TTSCP port. Don't be surprised if a bug found by vrfy turns out to be a false alarm because of a bug in vrfy itself.

No part of the vrfy TTSCP client should be mimicked by other software or be used as a study material. This client tries to be as ugly as possible and to crash any crashable server. Adding some ugly tests to this piece of code might raise the average code quality of Epos significantly.

7.5 More information

The header files mostly define basic interfaces for individual Epos components. Reading the ones related to a specific piece of code may often clarify things. Lots of global data declarations live in common.h; others (especially small, library-like functions) can be found in interf.h.

If you have any code or development related comment or question about Epos, send it to the Epos development mailing list epos@braille.mff.cuni.cz. You are also encouraged to subscribe to the list first by sending a mail containing only the text subscribe epos to epos.braille.mff.cuni.cz. Please spend a few seconds by trying to look up the answer in the documentation first.


Next Previous Contents