library_name: transformers
tags:
- Inductive
- Reasoning
language:
- en
base_model:
- meta-llama/Meta-Llama-3-8B-Instruct
pipeline_tag: text-generation
datasets:
- nsadeq/redis_generate_rule_alignment
- nsadeq/redis_generate_rule_sft
- nsadeq/redis_follow_rule_sft
Model Card for Model ID
ReDis-Llama is trained for improved inductive reasoning performance.
Model Description
- Developed by: Nafis Sadeq
- Language(s) (NLP): English
- Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct
Model Sources [optional]
- Repository: https://github.com/NafisSadeq/reasoning-distillation
- Paper: https://arxiv.org/abs/2504.10647
How to Get Started with the Model
Follow the instructions here: https://github.com/NafisSadeq/reasoning-distillation
Training Details
Training details can be found in the paper: https://arxiv.org/abs/2504.10647
Environmental Impact
- Hardware Type: 2 × 48 GB Nvidia RTX A6000 GPUs
- Hours used: 72 hours
Model Architecture and Objective
This model has the same architecture as meta-llama/Meta-Llama-3-8B-Instruct
Compute Infrastructure
2 × 48 GB Nvidia RTX A6000 GPUs
Citation
If you use this model, please cite the following paper.
@misc{sadeq2025improvingincontextlearningreasoning, title={Improving In-Context Learning with Reasoning Distillation}, author={Nafis Sadeq and Xin Xu and Zhouhang Xie and Julian McAuley and Byungkyu Kang and Prarit Lamba and Xiang Gao}, year={2025}, eprint={2504.10647}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2504.10647}, }