Weights Loaders¶
These classes perform the extraction, saving and loading of module weights to and from the file system.
All type-specific loader classes inherit from the common WeightsLoader base class which can also be extended
to add support for additional custom modules.
These classes provide the basis of adapter module integration into model classes such as adapter saving and loading. Depending on the model, one of these mixins should be implemented by every adapter-supporting model class.
WeightsLoader¶
-
class
transformers.WeightsLoader(model, weights_name, config_name)¶ An abstract class providing basic methods for saving and loading weights of a model. Extend this class to build custom module weight loaders.
-
abstract
filter_func(name: str) → Callable[str, bool]¶ The callable returned by this method is used to extract the module weights to be saved or loaded based on their names.
- Parameters
name (str) – An identifier of the weights to be saved.
- Returns
- A function that takes the fully qualified name of a module parameter and returns
a boolean value that specifies whether this parameter should be extracted.
- Return type
Callable[str, bool]
-
load(save_directory, load_as=None, loading_info=None, **kwargs) → Tuple[str, str]¶ Loads the module weights from the given directory. Override this method for additional loading actions. If adding the loaded weights to the model passed to the loader class requires adding additional modules, this method should also perform the architectural changes to the model.
- Parameters
save_directory (str) – The directory from where to load the weights.
load_as (str, optional) – Load the weights with this name. Defaults to None.
- Returns
- A tuple consisting of the local file system directory from which the weights where loaded
and the name of the loaded weights.
- Return type
Tuple[str, str]
-
abstract
rename_func(old_name: str, new_name: str) → Callable[str, str]¶ The callable returned by this method is used to optionally rename the module weights after loading.
- Parameters
old_name (str) – The string identifier of the weights as loaded from file.
new_name (str) – The new string identifier to which the weights should be renamed.
- Returns
- A function that takes the fully qualified name of a module parameter and returns
a new fully qualified name.
- Return type
Callable[str, str]
-
save(save_directory, name, **kwargs)¶ Saves the module config and weights into the given directory. Override this method for additional saving actions.
- Parameters
save_directory (str) – The directory to save the weights in.
name (str) – An identifier of the weights to be saved. The details are specified by the implementor.
-
abstract
AdapterLoader¶
-
class
transformers.AdapterLoader(model, adapter_type=None)¶ A class providing methods for saving and loading adapter modules from the Hub, the filesystem or a remote url.
Model classes passed to this loader must implement the ModelAdaptersMixin class.
-
filter_func(adapter_name)¶ The callable returned by this method is used to extract the module weights to be saved or loaded based on their names.
- Parameters
name (str) – An identifier of the weights to be saved.
- Returns
- A function that takes the fully qualified name of a module parameter and returns
a boolean value that specifies whether this parameter should be extracted.
- Return type
Callable[str, bool]
-
load(adapter_name_or_path, config=None, version=None, model_name=None, load_as=None, loading_info=None, **kwargs)¶ Loads a pre-trained pytorch adapter module from the local file system or a remote location.
- Parameters
adapter_name_or_path (str) – can be either: - the identifier of a pre-trained task adapter to be loaded from Adapter Hub - a path to a directory containing adapter weights saved using model.saved_adapter() - a URL pointing to a zip folder containing a saved adapter module
config (str, optional) – The requested configuration of the adapter.
version (str, optional) – The version of the adapter to be loaded.
model_name (str, optional) – The string identifier of the pre-trained model.
load_as (str, optional) – Load the adapter using this name. By default, the name with which the adapter was saved will be used.
- Returns
- A tuple consisting of the local file system directory from which the weights where loaded
and the name of the loaded weights.
- Return type
Tuple[str, str]
-
rename_func(old_name, new_name)¶ The callable returned by this method is used to optionally rename the module weights after loading.
- Parameters
old_name (str) – The string identifier of the weights as loaded from file.
new_name (str) – The new string identifier to which the weights should be renamed.
- Returns
- A function that takes the fully qualified name of a module parameter and returns
a new fully qualified name.
- Return type
Callable[str, str]
-
save(save_directory, name, meta_dict=None)¶ Saves an adapter and its configuration file to a directory, so that it can be reloaded using the load() method.
- Parameters
save_directory (str) – a path to a directory where the adapter will be saved
task_name (str) – the name of the adapter to be saved
-
AdapterFusionLoader¶
-
class
transformers.AdapterFusionLoader(model, error_on_missing=True)¶ A class providing methods for saving and loading AdapterFusion modules from the file system.
-
filter_func(adapter_fusion_name)¶ The callable returned by this method is used to extract the module weights to be saved or loaded based on their names.
- Parameters
name (str) – An identifier of the weights to be saved.
- Returns
- A function that takes the fully qualified name of a module parameter and returns
a boolean value that specifies whether this parameter should be extracted.
- Return type
Callable[str, bool]
-
load(save_directory, load_as=None, loading_info=None)¶ Loads a AdapterFusion module from the given directory.
- Parameters
save_directory (str) – The directory from where to load the weights.
load_as (str, optional) – Load the weights with this name. Defaults to None.
- Returns
- A tuple consisting of the local file system directory from which the weights where loaded
and the name of the loaded weights.
- Return type
Tuple[str, str]
-
rename_func(old_name, new_name)¶ The callable returned by this method is used to optionally rename the module weights after loading.
- Parameters
old_name (str) – The string identifier of the weights as loaded from file.
new_name (str) – The new string identifier to which the weights should be renamed.
- Returns
- A function that takes the fully qualified name of a module parameter and returns
a new fully qualified name.
- Return type
Callable[str, str]
-
save(save_directory: str, name: str)¶ Saves a AdapterFusion module into the given directory.
- Parameters
save_directory (str) – The directory to save the weights in.
name (str, optional) – The AdapterFusion name.
-
PredictionHeadLoader¶
-
class
transformers.PredictionHeadLoader(model, error_on_missing=True)¶ A class providing methods for saving and loading prediction head modules from the file system.
Model classes supporting configurable head modules via config files should provide a prediction head dict at model.heads and a method add_prediction_head(head_name, config).
-
filter_func(head_name)¶ The callable returned by this method is used to extract the module weights to be saved or loaded based on their names.
- Parameters
name (str) – An identifier of the weights to be saved.
- Returns
- A function that takes the fully qualified name of a module parameter and returns
a boolean value that specifies whether this parameter should be extracted.
- Return type
Callable[str, bool]
-
load(save_directory, load_as=None, loading_info=None)¶ Loads a prediction head module from the given directory.
- Parameters
save_directory (str) – The directory from where to load the weights.
load_as (str, optional) – Load the weights with this name. Defaults to None.
- Returns
- A tuple consisting of the local file system directory from which the weights where loaded
and the name of the loaded weights.
- Return type
Tuple[str, str]
-
rename_func(old_name, new_name)¶ The callable returned by this method is used to optionally rename the module weights after loading.
- Parameters
old_name (str) – The string identifier of the weights as loaded from file.
new_name (str) – The new string identifier to which the weights should be renamed.
- Returns
- A function that takes the fully qualified name of a module parameter and returns
a new fully qualified name.
- Return type
Callable[str, str]
-
save(save_directory: str, name: str = None)¶ Saves a prediction head module into the given directory.
- Parameters
save_directory (str) – The directory to save the weights in.
name (str, optional) – The prediction head name.
-