This is coded like this: | |
py | |
def _init_weights(self, module): | |
"""Initialize the weights""" | |
if isinstance(module, Wav2Vec2ForPreTraining): | |
module.project_hid.reset_parameters() | |
module.project_q.reset_parameters() | |
module.project_hid._is_hf_initialized = True | |
module.project_q._is_hf_initialized = True | |
elif isinstance(module, nn.Linear): | |
module.weight.data.normal_(mean=0.0, std=self.config.initializer_range) | |
if module.bias is not None: | |
module.bias.data.zero_() | |
The _is_hf_initialized flag is internally used to make sure we only initialize a submodule once. |