polytracker.grammars¶

grammars classes¶

Completion¶

class polytracker.grammars.Completion(prediction: Prediction, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int)¶

Bases: EarleyState

__init__(prediction: Prediction, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int)¶

add_predecessor(left_sibling: EarleyState)¶

completed_by: Set[EarleyState]¶

depth: int¶

expected: Tuple[Union[str, Terminal], ...]¶

property finished: bool¶

index: int¶

property next_element: Union[str, Terminal]¶

parsed: Tuple[Union[str, Terminal], ...]¶

predecessors: FrozenSet[EarleyState]¶

prediction: Prediction¶

property production: Production¶

to_tree() → MutableParseTree[Union[Production, Rule, Terminal]]¶

CorruptedGrammarError¶

class polytracker.grammars.CorruptedGrammarError¶

Bases: GrammarError

__init__(*args, **kwargs)¶

add_note()¶: Exception.add_note(note) – add a note to the exception

args¶

with_traceback()¶: Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

DisconnectedGrammarError¶

class polytracker.grammars.DisconnectedGrammarError¶

Bases: GrammarError

__init__(*args, **kwargs)¶

add_note()¶: Exception.add_note(note) – add a note to the exception

args¶

with_traceback()¶: Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

EarleyParser¶

class polytracker.grammars.EarleyParser(grammar: Grammar, sentence: Union[str, bytes], start: Optional[Production] = None)¶

Bases: object

__init__(grammar: Grammar, sentence: Union[str, bytes], start: Optional[Production] = None)¶

property end_states: Iterator[EarleyState]¶

parse() → Iterator[ParseTree[Union[Production, Rule, Terminal]]]¶

parse_trees() → Iterator[ParseTree[Union[Production, Rule, Terminal]]]¶: Reconstructs all parse trees from the parse

EarleyQueue¶

class polytracker.grammars.EarleyQueue(parser: EarleyParser)¶

Bases: object

__init__(parser: EarleyParser)¶

add(state: S, left_sibling: Optional[EarleyState] = None) → S¶

complete_state(state: EarleyState, completed: EarleyState)¶

remove(*states: Union[EarleyState, Iterable]) → int¶

EarleyState¶

class polytracker.grammars.EarleyState(prediction: Prediction, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int)¶

Bases: object

__init__(prediction: Prediction, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int)¶

add_predecessor(left_sibling: EarleyState)¶

depth: int¶

expected: Tuple[Union[str, Terminal], ...]¶

property finished: bool¶

index: int¶

property next_element: Union[str, Terminal]¶

parsed: Tuple[Union[str, Terminal], ...]¶

predecessors: FrozenSet[EarleyState]¶

prediction: Prediction¶

property production: Production¶

abstract to_tree() → MutableParseTree[Union[Production, Rule, Terminal]]¶

EmptyProduction¶

class polytracker.grammars.EmptyProduction(production: Production, prediction: Prediction, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int)¶

Bases: EarleyState

__init__(production: Production, prediction: Prediction, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int)¶

add_predecessor(left_sibling: EarleyState)¶

depth: int¶

expected: Tuple[Union[str, Terminal], ...]¶

property finished: bool¶

index: int¶

property next_element: Union[str, Terminal]¶

parsed: Tuple[Union[str, Terminal], ...]¶

predecessors: FrozenSet[EarleyState]¶

prediction: Prediction¶

property production: Production¶

to_tree() → MutableParseTree[Union[Production, Rule, Terminal]]¶

Grammar¶

class polytracker.grammars.Grammar¶

Bases: object

__init__()¶

dependency_graph() → DiGraph[Production]¶

find_partial_trees(sentence: bytes, start: Optional[Production] = None) → Iterator[ParseTree[Union[Production, Rule, Terminal]]]¶: Enumerates all partial parse trees that could result in the given starting sentence fragment.

load(raw_grammar: Dict[str, Any])¶

match(sentence: Union[str, bytes], start: Optional[Production] = None) → Match¶

remove(production: Union[str, Production]) → bool¶

simplify() → bool¶

verify(test_disconnection: bool = True)¶

GrammarError¶

class polytracker.grammars.GrammarError¶

Bases: RuntimeError

__init__(*args, **kwargs)¶

add_note()¶: Exception.add_note(note) – add a note to the exception

args¶

with_traceback()¶: Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

Match¶

class polytracker.grammars.Match(parser: EarleyParser)¶

Bases: object

__init__(parser: EarleyParser)¶

property parse_tree: Optional[ParseTree[Union[Production, Rule, Terminal]]]¶: Returns the first parse tree matched

MissingProductionError¶

class polytracker.grammars.MissingProductionError¶

Bases: CorruptedGrammarError

__init__(*args, **kwargs)¶

add_note()¶: Exception.add_note(note) – add a note to the exception

args¶

with_traceback()¶: Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

PartialMatch¶

class polytracker.grammars.PartialMatch(tree: ParseTree[Union[Production, Rule, Terminal]], remaining_symbols: Tuple[Union[str, Terminal], ...], remaining_bytes: bytes)¶

Bases: object

__init__(tree: ParseTree[Union[Production, Rule, Terminal]], remaining_symbols: Tuple[Union[str, Terminal], ...], remaining_bytes: bytes)¶

remaining_bytes: bytes¶

remaining_symbols: Tuple[Union[str, Terminal], ...]¶

tree: ParseTree[Union[Production, Rule, Terminal]]¶

Prediction¶

class polytracker.grammars.Prediction(production: Production, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int, rule: Rule)¶

Bases: EarleyState

__init__(production: Production, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int, rule: Rule)¶

add_predecessor(left_sibling: EarleyState)¶

depth: int¶

expected: Tuple[Union[str, Terminal], ...]¶

property finished: bool¶

index: int¶

property next_element: Union[str, Terminal]¶

parsed: Tuple[Union[str, Terminal], ...]¶

predecessors: FrozenSet[EarleyState]¶

prediction: Prediction¶

property production: Production¶

rule: Rule¶

to_tree() → MutableParseTree[Union[Production, Rule, Terminal]]¶

Production¶

class polytracker.grammars.Production(grammar: Grammar, name: str, *rules: Rule)¶

Bases: object

__init__(grammar: Grammar, name: str, *rules: Rule)¶

_propagate_terminals()¶: Calculates if this production can produce a terminal. If a rule calls another terminal whose _can_produce_terminal member is None, assume that it is False

add(rule: Rule) → bool¶

property can_produce_terminal: bool¶

first_rule() → Optional[Rule]¶

static load(grammar: Grammar, name: str, *rules: Iterable[str]) → Production¶

partial_match(sentence: bytes) → Iterator[PartialMatch]¶: Enumerates all partial parse trees and remaining symbols that match the given sentence

remove_recursive_rules() → Set[Rule]¶: Removes and returns all rules that solely recursively call this same production

remove_sub_production(name: str)¶

replace_sub_production(to_replace: str, replace_with: Union[str, Rule])¶

property used_by: Iterable[Production]¶

Rule¶

class polytracker.grammars.Rule(grammar: Grammar, *sequence: Union[str, Terminal])¶

Bases: object

__init__(grammar: Grammar, *sequence: Union[str, Terminal])¶

property can_produce_terminal: bool¶

static combine_terminals(sequence: Iterable[Union[str, Terminal]]) → Tuple[Union[str, Terminal], ...]¶

static load(grammar: Grammar, *sequence: Union[str, Terminal]) → Rule¶

remove_sub_production(prod_name: str) → bool¶

replace_sub_production(to_replace: str, replace_with: Union[str, Rule]) → bool¶

ScannedTerminal¶

class polytracker.grammars.ScannedTerminal(prediction: Prediction, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int, terminal: Terminal)¶

Bases: EarleyState

__init__(prediction: Prediction, parsed: Tuple[Union[str, Terminal], ...], expected: Tuple[Union[str, Terminal], ...], index: int, terminal: Terminal)¶

add_predecessor(left_sibling: EarleyState)¶

depth: int¶

expected: Tuple[Union[str, Terminal], ...]¶

property finished: bool¶

index: int¶

property next_element: Union[str, Terminal]¶

parsed: Tuple[Union[str, Terminal], ...]¶

predecessors: FrozenSet[EarleyState]¶

prediction: Prediction¶

property production: Production¶

terminal: Terminal¶

to_tree() → MutableParseTree[Union[Production, Rule, Terminal]]¶

grammars functions¶

extract¶

polytracker.grammars.extract(traces: Iterable[ProgramTrace], simplify: bool = False) → Grammar¶: extract a grammar from a set of traces

parse_tree_to_grammar¶

polytracker.grammars.parse_tree_to_grammar(tree: NonGeneralizedParseTree) → Grammar¶

production_name¶

polytracker.grammars.production_name(event: TraceEvent) → str¶

to_dot¶

polytracker.grammars.to_dot(graph: DiGraph, comment: Optional[str] = None) → Digraph¶

Parameters:: comment – comment for the graph
Returns:: Graphviz DiGraph

This function creates a dot object which can be saved to disk and converted to PDF its a visualization of the chain fragment, useful for visualizing a reorg.

trace_to_grammar¶

polytracker.grammars.trace_to_grammar(trace: ProgramTrace) → Grammar¶