--- library_name: transformers license: llama3 base_model: meta-llama/Meta-Llama-3-8B-Instruct tags: - llama-factory - full - generated_from_trainer model-index: - name: all_tasks_combined_8b_sft results: [] --- # all_tasks_combined_8b_sft This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on the identity and the data_mc_filtered datasets. It achieves the following results on the evaluation set: - Loss: 0.4943 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-05 - train_batch_size: 2 - eval_batch_size: 2 - seed: 42 - distributed_type: multi-GPU - num_devices: 4 - gradient_accumulation_steps: 4 - total_train_batch_size: 32 - total_eval_batch_size: 8 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 3.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:----:|:---------------:| | 0.4639 | 0.0929 | 50 | 0.5398 | | 0.4939 | 0.1857 | 100 | 0.5122 | | 0.4822 | 0.2786 | 150 | 0.5242 | | 0.4701 | 0.3714 | 200 | 0.5521 | | 0.4216 | 0.4643 | 250 | 0.5374 | | 0.4159 | 0.5571 | 300 | 0.5146 | | 0.4502 | 0.6500 | 350 | 0.5022 | | 0.4625 | 0.7428 | 400 | 0.4985 | | 0.4313 | 0.8357 | 450 | 0.4716 | | 0.4472 | 0.9285 | 500 | 0.4771 | | 0.2753 | 1.0204 | 550 | 0.5026 | | 0.2877 | 1.1133 | 600 | 0.4784 | | 0.3038 | 1.2061 | 650 | 0.4795 | | 0.2944 | 1.2990 | 700 | 0.4682 | | 0.2722 | 1.3918 | 750 | 0.4681 | | 0.2734 | 1.4847 | 800 | 0.4480 | | 0.2826 | 1.5775 | 850 | 0.4484 | | 0.2344 | 1.6704 | 900 | 0.4388 | | 0.2437 | 1.7632 | 950 | 0.4272 | | 0.2113 | 1.8561 | 1000 | 0.4233 | | 0.2548 | 1.9489 | 1050 | 0.4117 | | 0.1126 | 2.0409 | 1100 | 0.5031 | | 0.1128 | 2.1337 | 1150 | 0.4821 | | 0.0993 | 2.2266 | 1200 | 0.4997 | | 0.0978 | 2.3194 | 1250 | 0.4896 | | 0.1056 | 2.4123 | 1300 | 0.4980 | | 0.0897 | 2.5051 | 1350 | 0.4883 | | 0.0872 | 2.5980 | 1400 | 0.4941 | | 0.0916 | 2.6908 | 1450 | 0.4939 | | 0.0844 | 2.7837 | 1500 | 0.4945 | | 0.0959 | 2.8765 | 1550 | 0.4943 | | 0.094 | 2.9694 | 1600 | 0.4941 | ### Framework versions - Transformers 4.49.0 - Pytorch 2.5.1+cu124 - Datasets 3.2.0 - Tokenizers 0.21.0