Weight Transfer
#3
by
yukiarimo
- opened
Hello! That looks cool! Is it possible to take existing LLaMA 3.1 8B and LLaMA 3.2 1B models that I’ve already fine-tuned and adapt their weights, so they can be used with this code? Would appreciate some convert.py code. Thanks!
Thanks! You can use this notebook to convert the weights: https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/07_gpt_to_llama/standalone-llama32-mem-opt.ipynb
The only 2 changes you have to do are
In cell 18, replace the original Llama weight path with the path to your Llama weight file.
Then, at the end of the notebook, add:
torch.save(model.state_dict(), "my_weights.pth")