Class Token

The substring token of the original string with its interval location.

export

Hierarchy

Token

Index

Constructors

constructor

Properties

Constructors

constructor

new Token(value: string, interval: Interval): Token

- Defined in nlp/tokenizer.ts:42
Creates an instance of Token.

memberof

Token

Parameters
- value: string
  
  The substring.
- interval: Interval
  
  The interval location.
Returns Token

Properties

Readonly interval

interval: Interval

Readonly value

value: string

Generated using TypeDoc