Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
Note that, if one wants to
pre-train a model from scratch, one needs to either set the use_relative_position_bias or the
use_relative_position_bias attribute of [BeitConfig] to True in order to add
position embeddings.