raesl.compile.machine_files.builder

Code to construct line matching state machines.

The entry point to create a state machine is calling ‘create’ on a StateMachineBuilder instance.

A state machine has locations and edges.

One location in a machine may have an ‘initial’ option denoting it is the first location of the machine. Locations may also be accepting, denoting it is a proper end state. The accepting option takes a name, making it possible to distinguish which end location of the state machine was reached.

Edges start at a location and end at a location. Edges have a token-name, and may only be chosen when the input has a token with the same name. An edge may also have a tag option. The tag-name represents what kind of relevant token it is. When an edge is taken with a tag-name, the token matching with the edge is appended to a list associated with the tag-name. In this way, it is possible to extract relevant information from the matched sentence afterwards.

State machine execution assumes the machine is deterministic. From a location, each outgoing edge must have a different token-name. Note that a scanner may not map unique text to a token. The latter is resolved by sorting the edges from most specific token-name to least specific token-name.

In general, writing code to build locations and edges is bulky and hard to understand. Instead a text format is defined to allow writing the state machines as text, and let the computer build the state machine from the description. The text format accepted by the builder is:

state-machine ::=

MACHINE-NAME “:” { loc-def | edge-def }+

loc-def ::=

LOC-NAME [ “initial” ] [ “accept” “=” ACCEPT-name ] “;”

edge-def ::=

LOC-NAME “->” LOC-NAME “[” TOKEN-NAME “]” [ “tag” “=” TAG-NAME ] “;”

Locations in ‘loc-def’ must not exist already. Missing locations in ‘edge-def’ are silently created. Token names are defined in parsing/scanner.py.

Module Contents

Classes

StateMachineLexer	Lexer for tokenizing state machine descriptions.
StateMachineParser
ProcessingStateMachine	Extended StateMachine that also holds a callback function to add extracted
StateMachineBuilder	Class for easily constructing a state machine from a textual description.

class raesl.compile.machine_files.builder.StateMachineLexer

Bases: sly.Lexer

Lexer for tokenizing state machine descriptions.

Note that SLY and pylint are known not to like each other.

tokens

ignore = ' \t'

ignore_comment = '\\#.*'

NAME = '[A-Za-z][A-Za-z0-9_]*'

SQ_OPEN = '\\['

SQ_CLOSE = '\\]'

COMMA_TK = ','

SEMICOL_TK = ';'

EQ_TK = '='

COLON_TK = ':'

ARROW_TK = '->'

ignore_newline(t)

class raesl.compile.machine_files.builder.StateMachineParser

Bases: sly.Parser

tokens

error(t)

Default error handling function. This may be subclassed.

machine(p)

mlines(p)

mlines(p)

mlines(p)

loc_line(p)

loc_line(p)

loc_options(p)

loc_options(p)

loc_option(p)

loc_option(p)

edge_line(p)

edge_line(p)

edge_options(p)

edge_options(p)

edge_option(p)

class raesl.compile.machine_files.builder.ProcessingStateMachine(name: str, processing_func: raesl.compile.machine_files.typing.ProcessingFunc | None = None)

Bases: raesl.compile.state_machine.StateMachine

Extended StateMachine that also holds a callback function to add extracted parsing information into the ast.

Parameters:

• name – Name of the state machine, also the name of the matched sequence.

• processing_func – If not None, function that inserts relevant information from the matched line into the ast that is constructed.

class raesl.compile.machine_files.builder.StateMachineBuilder

Class for easily constructing a state machine from a textual description.

lexer

Lexer to tokenize the input text.

parser

Parser to interpret the tokens.

machine

State machine to be built.

locs

Temporary location map filled while building the state machine.

ensure_loc(name: str) → raesl.compile.state_machine.Location

Make sure a location with the provided name exists. If it doesn’t, create it.

Parameters:

name – Name of the location to ensure.

Returns:

The location with the provided name.

create_loc(name: str, opts: List[Tuple[str] | Tuple[str, str]]) → raesl.compile.state_machine.Location

Create a new location.

Parameters:

• name – Name of the location to create.

• opts – Location options, list of (‘initial’,) and/or (‘accept’, str) .

Returns:

The created location.

create(machine_text: str, processing_func: raesl.compile.machine_files.typing.ProcessingFunc | None = None) → ProcessingStateMachine

Create a state machine by interpreting the provided state machine text.

Parameters:

• machine_text – Description of the state machine as defined by StateMachineLexer and StateMachineParser.

• processing_func – Optional processing function to copy relevant information into the abstract syntax tree.

Returns:

The state machine that implements the description.