Model Mixins¶
These classes provide the basis of adapter module integration into model classes such as adapter saving and loading. Depending on the model, one of these mixins should be implemented by every adapter-supporting model class.
ModelAdaptersMixin¶
-
class
transformers.
ModelAdaptersMixin
(config, *args, **kwargs)¶ Mixin for transformer models adding support for loading/ saving adapters.
-
add_adapter
(adapter_name: str, adapter_type: transformers.adapter_utils.AdapterType, config=None)¶ Adds a new adapter module of the specified type to the model.
- Parameters
adapter_name (str) – The name of the adapter module to be added.
adapter_type (AdapterType) – The adapter type.
config (str or dict or AdapterConfig, optional) – The adapter configuration, can be either: - the string identifier of a pre-defined configuration dictionary - a configuration dictionary specifying the full config - if not given, the default configuration for this adapter type will be used
-
add_fusion
(adapter_names, adapter_fusion_config=None, override_kwargs=None)¶ Adds AdapterFusion to the model with alll the necessary configurations and weight initializations
- Parameters
adapter_names – a list of adapter names which should be fused
adapter_fusion_config (str or dict) – adapter fusion configuration, can be either: - a string identifying a pre-defined adapter fusion configuration - a dictionary representing the adapter fusion configuration - the path to a file containing the adapter fusion configuration
override_kwargs – dictionary items for values which should be overwritten in the default AdapterFusion configuration
-
freeze_model
(freeze=True)¶ Freezes all weights of the model.
-
load_adapter
(adapter_name_or_path: str, adapter_type: transformers.adapter_utils.AdapterType = None, config: Union[dict, str] = None, version: str = None, model_name: str = None, load_as: str = None, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None, **kwargs) → str¶ Loads a pre-trained pytorch adapter module from the local file system or a remote location.
- Parameters
adapter_name_or_path (str) – can be either: - the identifier of a pre-trained task adapter to be loaded from Adapter Hub - a path to a directory containing adapter weights saved using model.saved_adapter() - a URL pointing to a zip folder containing a saved adapter module
adapter_type (AdapterType, optional) – The type of adapter to be loaded. If not specified, text_task will be used for adapters loaded from the Hub.
config (dict or str, optional) – The requested configuration of the adapter. If not specified, will be either: - the default adapter config for the requested adapter if specified - the global default adapter config
version (str, optional) – The version of the adapter to be loaded.
model_name (str, optional) – The string identifier of the pre-trained model.
load_as (str, optional) – Load the adapter using this name. By default, the name with which the adapter was saved will be used.
- Returns
The name with which the adapter was added to the model.
- Return type
str
-
load_adapter_fusion
(adapter_fusion_name_or_path: str, load_as: str = None, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None, **kwargs) → str¶ Loads a pre-trained pytorch adapter module from the local file system or a remote location.
- Parameters
adapter_fusion_name_or_path (str) – can be either: - the identifier of a pre-trained task adapter fusion module to be loaded from Adapter Hub - a path to a directory containing adapter weights saved using model.saved_adapter() - a URL pointing to a zip folder containing a saved adapter module
config (dict or str, optional) – The requested configuration of the adapter fusion. If not specified, will be either: - the default adapter config for the requested adapter fusion if specified - the global default adapter fusion config
model_name (str, optional) – The string identifier of the pre-trained model.
load_as (str, optional) – Load the adapter using this name. By default, the name with which the adapter was saved will be used.
- Returns
The name with which the adapter was added to the model.
- Return type
str
-
save_adapter
(save_directory: str, adapter_name: str, meta_dict: dict = None, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None)¶ Saves an adapter and its configuration file to a directory so that it can be shared or reloaded using load_adapter().
- Parameters
save_directory (str) – Path to a directory where the adapter should be saved.
adapter_name (str) – Name of the adapter to be saved.
- Raises
ValueError – If the given adapter name is invalid.
-
save_adapter_fusion
(save_directory: str, adapter_names: list, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None)¶ Saves an adapter and its configuration file to a directory so that it can be shared or reloaded using load_adapter().
- Parameters
save_directory (str) – Path to a directory where the adapter should be saved.
adapter_name (str) – Name of the adapter to be saved.
- Raises
ValueError – If the given adapter name is invalid.
-
save_all_adapter_fusions
(save_directory: str, meta_dict: dict = None, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None)¶ Saves all adapters of this model together with their configuration to subfolders of the given location.
- Parameters
save_directory (str) – Path to a directory where the adapters should be saved.
-
save_all_adapters
(save_directory: str, meta_dict: dict = None, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None)¶ Saves all adapters of this model together with their configuration to subfolders of the given location.
- Parameters
save_directory (str) – Path to a directory where the adapters should be saved.
-
set_active_adapters
(adapter_names: list)¶ Sets the adapter modules to be used by default in every forward pass. This setting can be overriden by passing the adapter_names parameter in the foward() pass. If no adapter with the given name is found, no module of the respective type will be activated.
- Parameters
adapter_names (list) – The list of adapters to be activated by default. Can be a fusion or stacking configuration.
-
set_adapter_config
(adapter_type: transformers.adapter_utils.AdapterType, adapter_config)¶ Sets the adapter configuration of the specified adapter type.
- Parameters
adapter_type (AdapterType) – The adapter type.
adapter_config (str or dict) – adapter configuration, can be either: - a string identifying a pre-defined adapter configuration - a dictionary representing the adapter configuration - the path to a file containing the adapter configuration
-
set_adapter_fusion_config
(adapter_fusion_config, override_kwargs=None)¶ Sets the adapter fusion configuration.
- Parameters
adapter_fusion_config (str or dict) – adapter fusion configuration, can be either: - a string identifying a pre-defined adapter fusion configuration - a dictionary representing the adapter fusion configuration - the path to a file containing the adapter fusion configuration
-
abstract
train_adapter
(adapter_names: list)¶ Sets the model into mode for training the given adapters.
-
abstract
train_fusion
(adapter_names: list)¶ Sets the model into mode for training of adapter fusion determined by a list of adapter names.
-
ModelWithHeadsAdaptersMixin¶
-
class
transformers.
ModelWithHeadsAdaptersMixin
(config, *args, **kwargs)¶ Mixin adding support for loading/ saving adapters to transformer models with head(s).
-
add_adapter
(adapter_name: str, adapter_type: transformers.adapter_utils.AdapterType, config=None)¶ Adds a new adapter module of the specified type to the model.
- Parameters
adapter_name (str) – The name of the adapter module to be added.
adapter_type (AdapterType) – The adapter type.
config (str or dict, optional) – The adapter configuration, can be either: - the string identifier of a pre-defined configuration dictionary - a configuration dictionary specifying the full config - if not given, the default configuration for this adapter type will be used
-
load_adapter
(adapter_name_or_path: str, adapter_type: transformers.adapter_utils.AdapterType = None, config: Union[dict, str] = None, version: str = None, model_name: str = None, load_as: str = None, with_head: bool = True, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None, **kwargs) → str¶ Loads a pre-trained pytorch adapter module from the local file system or a remote location.
- Parameters
adapter_name_or_path (str) – can be either: - the identifier of a pre-trained task adapter to be loaded from Adapter Hub - a path to a directory containing adapter weights saved using model.saved_adapter() - a URL pointing to a zip folder containing a saved adapter module
adapter_type (AdapterType, optional) – The type of adapter to be loaded. If not specified, text_task will be used for adapters loaded from the Hub.
config (dict or str, optional) – The requested configuration of the adapter. If not specified, will be either: - the default adapter config for the requested adapter if specified - the global default adapter config
version (str, optional) – The version of the adapter to be loaded.
model_name (str, optional) – The string identifier of the pre-trained model.
load_as (str, optional) – Load the adapter using this name. By default, the name with which the adapter was saved will be used.
- Returns
The name with which the adapter was added to the model.
- Return type
str
-
save_adapter
(save_directory: str, adapter_name: str, with_head: bool = True, meta_dict: dict = None, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None)¶ Saves an adapter and its configuration file to a directory so that it can be shared or reloaded using load_adapter().
- Parameters
save_directory (str) – Path to a directory where the adapter should be saved.
adapter_name (str) – Name of the adapter to be saved.
- Raises
ValueError – If the given adapter name is invalid.
-
save_all_adapters
(save_directory: str, with_head: bool = True, meta_dict: dict = None, custom_weights_loaders: Optional[List[transformers.adapter_model_mixin.WeightsLoader]] = None)¶ Saves all adapters of this model together with their configuration to subfolders of the given location.
- Parameters
save_directory (str) – Path to a directory where the adapters should be saved.
-
train_adapter
(adapter_names: list)¶ Sets the model into mode for training the given adapters.
-
train_fusion
(adapter_names: list)¶ Sets the model in mode for training of adapter fusion determined by a list of adapter names.
-