Adapter Utilities¶
A collection of utility methods mainly related to searching and loading adapter modules from Adapter-Hub.
-
class
transformers.adapter_utils.
AdapterType
(value)¶ Models all currently available model adapter types.
-
class
transformers.adapter_utils.
DataclassJSONEncoder
(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)¶ -
default
(o)¶ Implement this method in a subclass such that it returns a serializable object for
o
, or calls the base implementation (to raise aTypeError
).For example, to support arbitrary iterators, you could implement default like this:
def default(self, o): try: iterable = iter(o) except TypeError: pass else: return list(iterable) # Let the base class default method raise the TypeError return JSONEncoder.default(self, o)
-
-
transformers.adapter_utils.
get_adapter_config_hash
(config, length=16)¶ Calculates the hash of a given adapter configuration which is used to identify this configuration.
- Returns
The resulting hash of the given config dict.
- Return type
str
-
transformers.adapter_utils.
pull_from_hub
(specifier: str, adapter_type: transformers.adapter_utils.AdapterType, model_name: str, adapter_config: Optional[Union[dict, str]] = None, version: str = None, strict: bool = False, **kwargs) → str¶ Downloads a pre-trained adapter module from Adapter-Hub
- Parameters
specifier (str) – A string specifying the adapter to be loaded.
adapter_type (AdapterType) – The adapter type.
model_name (str) – The identifier of the pre-trained model for which to load an adapter.
adapter_config (Union[dict, str], optional) – The configuration of the adapter to be loaded.
version (str, optional) – The version of the adapter to be loaded. Defaults to None.
strict (bool, optional) – If set to True, only allow adapters exactly matching the given config to be loaded. Defaults to False.
- Returns
The local path to which the adapter has been downloaded.
- Return type
str
-
transformers.adapter_utils.
resolve_adapter_config
(config: Union[dict, str], local_map=None, try_loading_from_hub=True, **kwargs) → dict¶ Resolves a given adapter configuration specifier to a full configuration dictionary.
- Parameters
config (Union[dict, str]) –
The configuration to resolve. Can be either:
a dictionary: returned without further action
an identifier string available in local_map
the path to a file containing a full adapter configuration
an identifier string available in Adapter-Hub
- Returns
The resolved adapter configuration dictionary.
- Return type
dict
-
transformers.adapter_utils.
resolve_adapter_path
(adapter_name_or_path, adapter_type: transformers.adapter_utils.AdapterType = text_task, model_name: str = None, adapter_config: Union[dict, str] = None, version: str = None, **kwargs) → str¶ Resolves the path to a pre-trained adapter module. Note: If attempting to resolve an adapter from the Hub, adapter_config, adapter_type and model_name must be present.
- Parameters
adapter_name_or_path (str) –
Can be either:
the path to a folder in the file system containing the adapter configuration and weights
an url pointing to a zip folder containing the adapter configuration and weights
a specifier matching a pre-trained adapter uploaded to Adapter-Hub
adapter_type (AdapterType, optional) – The adapter type.
model_name (str, optional) – The identifier of the pre-trained model for which to load an adapter.
adapter_config (Union[dict, str], optional) – The configuration of the adapter to be loaded.
version (str, optional) – The version of the adapter to be loaded. Defaults to None.
- Returns
The local path from where the adapter module can be loaded.
- Return type
str