A "manual" provider of tokens.

interface TokensProvider {
    getInitialState(): IState;
    tokenize(line, state): ILineTokens;
}

Methods

  • The initial state of a language. Will be the state passed in to tokenize the first line.

    Returns IState

  • Tokenize a line given the state at the beginning of the line.

    Parameters

    Returns ILineTokens