Package Mercury.Scanning

Classes
interface	ITokenizer
	The tokenizer interface. More...
class	Token
	Token, a part of an input text recognized by tokenizer More...
class	Tokenizer
	A default implementation of ITokenizer interface using Microsoft Regular Expressions. Splits tokens in the following fashion: More...

Enumerations
enum	TokenType { TokenType.String, TokenType.Symbol, TokenType.Keyword, TokenType.Number }
	Defines token types. The actual conditions depend on the Mercury.Scanning.ITokenizer implementation. More...
enum	TokenizerOptions {   TokenizerOptions.None = 0, TokenizerOptions.CompiledMatch = 1, TokenizerOptions.IgnoreNumbers = 2, TokenizerOptions.IgnoreCase = 4,   TokenizerOptions.IgnoreQuotes = 8, TokenizerOptions.IgnoreSigns = 16, TokenizerOptions.StrictNumbers = 32 }
	Options for the tokenizer More...

Enumeration Type Documentation

enum Mercury.Scanning.TokenizerOptions

Options for the tokenizer

Enumerator
None	summary>No options summary>Compile regex pattern, which yields faster matching at the cost of slower initialization.
CompiledMatch	summary>Will not detect numbers, but consider them strings instead.
IgnoreNumbers	summary>Ignores case when matching patterns.
IgnoreCase	summary>Treats quote characters as symbols.
IgnoreQuotes	summary>Detects only positive numbers.
IgnoreSigns	summary>With this option, numbers must be separated by spaces.
StrictNumbers

Definition at line 26 of file Tokenizer.cs.

enum Mercury.Scanning.TokenType

Defines token types. The actual conditions depend on the Mercury.Scanning.ITokenizer implementation.

Enumerator
String	summary>String. summary>Symbol.
Symbol	summary>Keyword (a reserved word).
Keyword	summary>Number.
Number

Definition at line 12 of file Token.cs.