plugins.hf_ner.implementation

HfNERPlugin Objects

class HfNERPlugin(Plugin)

Named Entity Recognition or Token Classification plugin for HuggingFace models

plugin.setup() bootstraps the entire pipeline and returns a fully setup trainer.

Example:

trainer = plugin.setup_trainer() trainer.train() trainer.validate()

__init__

def __init__(config: Optional[Dict] = None)

CustomArgParser parses YAML config located at cmdline --config_path. If --config_path is not provided, assumes YAML file is named config.yaml and present in working directory. Instantiates dataclasses: self.data_args (arguments.DataInterfaceArguments): Instantiated dataclass containing args required to initialize NERDataInterface and NERProcessor classes self.module_args (arguments.ModuleInterfaceArguments): Instantiated dataclass containing args required to initialize NERModule class

Sets properties: self.datainterface: data_interface.DataInterface [NERDataInterface] object self.dataprocessor: data_interface.DataProcessor [NERProcessor] object. These two together are used to read raw data and create sequences of tokens in setup_datainterface. The processed data is fed to HuggingFace AutoModelForTokenClassification models. self.module: module_interface.ModuleInterface [NERModule] object This is used to initialize a Marlin trainer.