Adapter Modules

Classes implementing task and language adapters.

class transformers.adapter_modeling.Activation_Function_Class(hidden_act)

Implementation of various activation function.

class transformers.adapter_modeling.Adapter(input_size, down_sample=None, non_linearity='relu', init_bert_weights=True, add_layer_norm_before=True, add_layer_norm_after=False, residual_before_ln=True)

Implementation of a single Adapter block.

static init_bert_weights(module)

Initialize the weights.

class transformers.adapter_modeling.AdapterFusionSentLvlDynamic(config, n_tasks)
class transformers.adapter_modeling.BertFusion(config)
class transformers.adapter_modeling.GLOWCouplingBlock(dims_in, dims_c=[], non_linearity='relu', reduction_factor=2, clamp=5.0)

Coupling Block following the GLOW design. The only difference to the RealNVP coupling blocks, is the fact that it uses a single subnetwork to jointly predict [s_i, t_i], instead of two separate subnetworks. This reduces computational cost and speeds up learning. clamp: Soft clamping for the multiplicative component. The amplification or attenuation

of each input dimension can be at most ±exp(clamp).

class transformers.adapter_modeling.NICECouplingBlock(dims_in, dims_c=[], non_linearity='relu', reduction_factor=2)

Coupling Block following the NICE design.