Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
raw
history blame contribute delete
628 Bytes
This is coded like this:
py
def _init_weights(self, module):
"""Initialize the weights"""
if isinstance(module, Wav2Vec2ForPreTraining):
module.project_hid.reset_parameters()
module.project_q.reset_parameters()
module.project_hid._is_hf_initialized = True
module.project_q._is_hf_initialized = True
elif isinstance(module, nn.Linear):
module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)
if module.bias is not None:
module.bias.data.zero_()
The _is_hf_initialized flag is internally used to make sure we only initialize a submodule once.