Mercury.Scanning.ITokenizer Interface Reference

The tokenizer interface. More...

Inheritance diagram for Mercury.Scanning.ITokenizer:

Public Member Functions
IList< Token >	Tokenize (string input)
	Splits source string into tokens. More...

Detailed Description

The tokenizer interface.

Definition at line 14 of file Tokenizer.cs.

Member Function Documentation

IList<Token> Mercury.Scanning.ITokenizer.Tokenize

(

string

input

)

Splits source string into tokens.

Parameters

input

Source string.

Returns

List of tokens.

Implemented in Mercury.Scanning.Tokenizer.

---

The documentation for this interface was generated from the following file:

• Mercury/Mercury/Scanning/Tokenizer.cs