Pipeline and Tokenizer Loading Issue on Hugging Face Hub
Hello, I have two questions to ask. First, I have the impression that when calling the model via the pipeline function, it doesn't load the tokenizer. Is this expected behavior, or do you know why this occurs?
Secondly, I notice that the initial training is in 32 bits, yet in the configuration file, the torch_dtype parameter is set to bfloat16. I think this could lead to an error if torch_dtype is set to "auto", causing the model to load in bfloat16. Is this intended behavior?
Hey @Cyrile , we didn’t spend time instantiating the pipeline function, so I can’t confirm its behavior yet. We plan to submit our paper first, then return to model development (including the pipeline function, NER, etc.). For now, you may need to instantiate the tokenizer manually.
Regarding torch_dtype, you are absolutely right; we modified the config file. We started with the Llama config, which is set to bfloat16, but in our case, this was an error. Thanks a lot for raising this issue!