Module static_analyser::lex [] [src]

Module for performing lexical analysis on source code.

Structs

Token

A valid Delphi source code token.

Enums

TokenKind

Any valid token in the Delphi programming language.

Functions

tokenize

Turn a string of valid Delphi code into a list of tokens, including the location of that token's start and end point in the original source code.

tokenize_single_token

Try to lex a single token from the input stream.