plugins.plugin_module_interface

plugin module interface

PluginModuleInterface Objects

class PluginModuleInterface(module_interface.ModuleInterface)

Common plugin module interface to easily load Huggingface tokenizers and Configs

auto_setup

def auto_setup(automodel_class)

Run all (tokenizer,config,model) setups

data

@property

def data()

DataInterface object that is used to retrieve corresponding train or val dataset.

Returns:

• data - DataInterface object with at least one of train or val data.

model

@property

def model()

Pytorch model.

setup_tokenizer

def setup_tokenizer()

Initializes AutoTokenizer from model_args.tokenizer_path or model_args.hf_model string

setup_model_config

def setup_model_config()

Initializes AutoConfig from model_args.model_config + model_args.model_config_file path or model_args.hf_model string

setup_model

def setup_model(automodel_class)

Initializes automodel_class arg by either: Option 1: Load weights from specified files mentioned in YAML config model: model_config_path model_config_file model_path model_file Option 2: Load from Huggingface model hub, specify string in YAML config as: model: hf_model

Arguments:

• automodel_class - Huggingface AutoModelFor* class