Weights Loaders

These classes perform the extraction, saving and loading of module weights to and from the file system. All type-specific loader classes inherit from the common WeightsLoader base class which can also be extended to add support for additional custom modules.

These classes provide the basis of adapter module integration into model classes such as adapter saving and loading. Depending on the model, one of these mixins should be implemented by every adapter-supporting model class.

WeightsLoader

class transformers.WeightsLoader(model, weights_name, config_name)

An abstract class providing basic methods for saving and loading weights of a model. Extend this class to build custom module weight loaders.

abstract filter_func(name: str) → Callable[str, bool]

The callable returned by this method is used to extract the module weights to be saved or loaded based on their names.

Parameters

name (str) – An identifier of the weights to be saved.

Returns

A function that takes the fully qualified name of a module parameter and returns

a boolean value that specifies whether this parameter should be extracted.

Return type

Callable[str, bool]

load(save_directory, load_as=None, loading_info=None, **kwargs) → Tuple[str, str]

Loads the module weights from the given directory. Override this method for additional loading actions. If adding the loaded weights to the model passed to the loader class requires adding additional modules, this method should also perform the architectural changes to the model.

Parameters
  • save_directory (str) – The directory from where to load the weights.

  • load_as (str, optional) – Load the weights with this name. Defaults to None.

Returns

A tuple consisting of the local file system directory from which the weights where loaded

and the name of the loaded weights.

Return type

Tuple[str, str]

abstract rename_func(old_name: str, new_name: str) → Callable[str, str]

The callable returned by this method is used to optionally rename the module weights after loading.

Parameters
  • old_name (str) – The string identifier of the weights as loaded from file.

  • new_name (str) – The new string identifier to which the weights should be renamed.

Returns

A function that takes the fully qualified name of a module parameter and returns

a new fully qualified name.

Return type

Callable[str, str]

save(save_directory, name, **kwargs)

Saves the module config and weights into the given directory. Override this method for additional saving actions.

Parameters
  • save_directory (str) – The directory to save the weights in.

  • name (str) – An identifier of the weights to be saved. The details are specified by the implementor.

AdapterLoader

class transformers.AdapterLoader(model, adapter_type=None)

A class providing methods for saving and loading adapter modules from the Hub, the filesystem or a remote url.

Model classes passed to this loader must implement the ModelAdaptersMixin class.

filter_func(adapter_name)

The callable returned by this method is used to extract the module weights to be saved or loaded based on their names.

Parameters

name (str) – An identifier of the weights to be saved.

Returns

A function that takes the fully qualified name of a module parameter and returns

a boolean value that specifies whether this parameter should be extracted.

Return type

Callable[str, bool]

load(adapter_name_or_path, config=None, version=None, model_name=None, load_as=None, loading_info=None, **kwargs)

Loads a pre-trained pytorch adapter module from the local file system or a remote location.

Parameters
  • adapter_name_or_path (str) – can be either: - the identifier of a pre-trained task adapter to be loaded from Adapter Hub - a path to a directory containing adapter weights saved using model.saved_adapter() - a URL pointing to a zip folder containing a saved adapter module

  • config (str, optional) – The requested configuration of the adapter.

  • version (str, optional) – The version of the adapter to be loaded.

  • model_name (str, optional) – The string identifier of the pre-trained model.

  • load_as (str, optional) – Load the adapter using this name. By default, the name with which the adapter was saved will be used.

Returns

A tuple consisting of the local file system directory from which the weights where loaded

and the name of the loaded weights.

Return type

Tuple[str, str]

rename_func(old_name, new_name)

The callable returned by this method is used to optionally rename the module weights after loading.

Parameters
  • old_name (str) – The string identifier of the weights as loaded from file.

  • new_name (str) – The new string identifier to which the weights should be renamed.

Returns

A function that takes the fully qualified name of a module parameter and returns

a new fully qualified name.

Return type

Callable[str, str]

save(save_directory, name, meta_dict=None)

Saves an adapter and its configuration file to a directory, so that it can be reloaded using the load() method.

Parameters
  • save_directory (str) – a path to a directory where the adapter will be saved

  • task_name (str) – the name of the adapter to be saved

AdapterFusionLoader

class transformers.AdapterFusionLoader(model, error_on_missing=True)

A class providing methods for saving and loading AdapterFusion modules from the file system.

filter_func(adapter_fusion_name)

The callable returned by this method is used to extract the module weights to be saved or loaded based on their names.

Parameters

name (str) – An identifier of the weights to be saved.

Returns

A function that takes the fully qualified name of a module parameter and returns

a boolean value that specifies whether this parameter should be extracted.

Return type

Callable[str, bool]

load(save_directory, load_as=None, loading_info=None)

Loads a AdapterFusion module from the given directory.

Parameters
  • save_directory (str) – The directory from where to load the weights.

  • load_as (str, optional) – Load the weights with this name. Defaults to None.

Returns

A tuple consisting of the local file system directory from which the weights where loaded

and the name of the loaded weights.

Return type

Tuple[str, str]

rename_func(old_name, new_name)

The callable returned by this method is used to optionally rename the module weights after loading.

Parameters
  • old_name (str) – The string identifier of the weights as loaded from file.

  • new_name (str) – The new string identifier to which the weights should be renamed.

Returns

A function that takes the fully qualified name of a module parameter and returns

a new fully qualified name.

Return type

Callable[str, str]

save(save_directory: str, name: str)

Saves a AdapterFusion module into the given directory.

Parameters
  • save_directory (str) – The directory to save the weights in.

  • name (str, optional) – The AdapterFusion name.

PredictionHeadLoader

class transformers.PredictionHeadLoader(model, error_on_missing=True)

A class providing methods for saving and loading prediction head modules from the file system.

Model classes supporting configurable head modules via config files should provide a prediction head config at model.config.prediction_heads and a method add_prediction_head(head_name, config).

filter_func(head_name)

The callable returned by this method is used to extract the module weights to be saved or loaded based on their names.

Parameters

name (str) – An identifier of the weights to be saved.

Returns

A function that takes the fully qualified name of a module parameter and returns

a boolean value that specifies whether this parameter should be extracted.

Return type

Callable[str, bool]

load(save_directory, load_as=None, loading_info=None)

Loads a prediction head module from the given directory.

Parameters
  • save_directory (str) – The directory from where to load the weights.

  • load_as (str, optional) – Load the weights with this name. Defaults to None.

Returns

A tuple consisting of the local file system directory from which the weights where loaded

and the name of the loaded weights.

Return type

Tuple[str, str]

rename_func(old_name, new_name)

The callable returned by this method is used to optionally rename the module weights after loading.

Parameters
  • old_name (str) – The string identifier of the weights as loaded from file.

  • new_name (str) – The new string identifier to which the weights should be renamed.

Returns

A function that takes the fully qualified name of a module parameter and returns

a new fully qualified name.

Return type

Callable[str, str]

save(save_directory: str, name: str = None)

Saves a prediction head module into the given directory.

Parameters
  • save_directory (str) – The directory to save the weights in.

  • name (str, optional) – The prediction head name.

WeightsLoaderHelper

class transformers.WeightsLoaderHelper(model, weights_name, config_name)

A class providing helper methods for saving and loading module weights.