Tags and Tokens

Overview

Data that defines the rule constructs of a Class, Race, Language, and other characteristics and attributes of a PlayerCharacter are stored in LST files. These LST files are parsed to import the information from those LST files into objects within the PCGen program.

For purposes of the architecture descriptions here on the Wiki, the individual entries in the data persistence files will be referred to as "Tags". To store information, these tags separate the "key" (used to indicate the tag name) and the "value" (the information the tag is conveying to PCGen) by a colon. A simple example can be taken from the Common Language in the Language LST file from SRD 3.0:

TYPE:Spoken

This indicates the "TYPE" tag is indicating a value of "Spoken".

For purposes of this document, the code that parses the value of a tag and translates the tag syntax into the internal data structure of PCGen is called a Token.

Occasionally, you fill find these the terms "tag" and "token" used differently; which name is used will likely depend on the perspective of the speaker. The Wiki makes effort to use the terms Tag and Token in the context they are defined in this section. This is a decision for clarity, and is not a judgement of the "best" method for referring to the items in the data persistence format or the code.

Types of tokens (meaning the Interfaces that describe the behavior for each type of token) are located in pcgen.rules.persistence.token.

Tokens that implement these interfaces, which are effectively extensions to the Rules Persistence System, are found in the plugin.lsttokens package.

Types of Tokens

CDOMToken

A CDOMToken is a PCGen 5.16 token that has a parse() method for processing the value of a tag and a getTokenName() method to identify the name of the tag the Token can process. Note that a CDOMToken does not require unparse(), as compatibility tokens from previous revisions would not implement that behavior.

CDOMPrimaryToken

A CDOMPrimaryToken is a CDOMToken that is active (not deprecated) in the current version of PCGen, and thus implements the unparse() method.

CDOMSubToken

A CDOMSubToken is a CDOMToken that is designed to operate for Subtokens

CDOMSecondaryToken

A CDOMSecondaryToken is a CDOMSubToken that is active (not deprecated) in the current version of PCGen, and thus implements the unparse() method.

CDOMCompatibilityToken

A CDOMCompatibilityToken is a CDOMToken that implements a tag syntax from a previous version of PCGen, see Token Compatibility. Once deprecated, a CDOMPrimaryToken would become a CDOMCompatibilityToken.

CDOMCompatibilitySubToken

A CDOMCompatibilitySubToken is a CDOMSubToken that implements a tag syntax from a previous version of PCGen, see Token Compatibility. Once deprecated, a CDOMSecondaryToken would become a CDOMCompatibilitySubToken.

ClassWrappedToken

A ClassWrappedToken is a specialized CDOMToken designed to provide compatibility for using CLASS tokens on the level 1 CLASSLEVEL line, see Weaknesses of Class Wrapped Tokens.

PreCompatibilityToken

A PreCompatibilityToken is a token for wrapping the PCGen 5.14-type Prerequisite tokens into 5.16-type tokens, see Weaknesses of Prerequisite Tokens.

Note about Token Behavior

A Note of Caution on Tags/Tokens: Different tags have different behavior with respect to overwriting or appending values. Some tags may only allow a single value (e.g. RACETYPE from Race LST files), and subsequent references to that tag in the same object will overwrite the original value. Other tags are effectively lists (e.g. the Global TYPE tag), and subsequent calls to that tag will append values to the list. The behavior is noted within the tag documentation distributed with PCGen.