Adapter Utilities

A collection of utility methods mainly related to searching and loading adapter modules from Adapter-Hub.

class transformers.adapter_utils.AdapterType(value)

Models all currently available model adapter types.

class transformers.adapter_utils.DataclassJSONEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)
default(o)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)
transformers.adapter_utils.get_adapter_config_hash(config, length=16)

Calculates the hash of a given adapter configuration which is used to identify this configuration.

Returns

The resulting hash of the given config dict.

Return type

str

transformers.adapter_utils.pull_from_hub(specifier: str, adapter_type: transformers.adapter_utils.AdapterType, model_name: str, adapter_config: Optional[Union[dict, str]] = None, version: str = None, strict: bool = False, **kwargs) → str

Downloads a pre-trained adapter module from Adapter-Hub

Parameters

  • specifier (str) – A string specifying the adapter to be loaded.

  • adapter_type (AdapterType) – The adapter type.

  • model_name (str) – The identifier of the pre-trained model for which to load an adapter.

  • adapter_config (Union[dict, str], optional) – The configuration of the adapter to be loaded.

  • version (str, optional) – The version of the adapter to be loaded. Defaults to None.

  • strict (bool, optional) – If set to True, only allow adapters exactly matching the given config to be loaded. Defaults to False.

Returns

The local path to which the adapter has been downloaded.

Return type

str

transformers.adapter_utils.resolve_adapter_config(config: Union[dict, str], local_map=None, try_loading_from_hub=True, **kwargs) → dict

Resolves a given adapter configuration specifier to a full configuration dictionary.

Parameters

config (Union[dict, str]) –

The configuration to resolve. Can be either:

  • a dictionary: returned without further action

  • an identifier string available in local_map

  • the path to a file containing a full adapter configuration

  • an identifier string available in Adapter-Hub

Returns

The resolved adapter configuration dictionary.

Return type

dict

transformers.adapter_utils.resolve_adapter_path(adapter_name_or_path, adapter_type: transformers.adapter_utils.AdapterType = text_task, model_name: str = None, adapter_config: Union[dict, str] = None, version: str = None, **kwargs) → str

Resolves the path to a pre-trained adapter module. Note: If attempting to resolve an adapter from the Hub, adapter_config, adapter_type and model_name must be present.

Parameters

  • adapter_name_or_path (str) –

    Can be either:

    • the path to a folder in the file system containing the adapter configuration and weights

    • an url pointing to a zip folder containing the adapter configuration and weights

    • a specifier matching a pre-trained adapter uploaded to Adapter-Hub

  • adapter_type (AdapterType, optional) – The adapter type.

  • model_name (str, optional) – The identifier of the pre-trained model for which to load an adapter.

  • adapter_config (Union[dict, str], optional) – The configuration of the adapter to be loaded.

  • version (str, optional) – The version of the adapter to be loaded. Defaults to None.

Returns

The local path from where the adapter module can be loaded.

Return type

str