|
|
--- |
|
|
library_name: transformers |
|
|
tags: |
|
|
- Inductive |
|
|
- Reasoning |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- meta-llama/Meta-Llama-3-8B-Instruct |
|
|
pipeline_tag: text-generation |
|
|
datasets: |
|
|
- nsadeq/redis_generate_rule_alignment |
|
|
- nsadeq/redis_generate_rule_sft |
|
|
- nsadeq/redis_follow_rule_sft |
|
|
--- |
|
|
|
|
|
# Model Card for Model ID |
|
|
|
|
|
ReDis-Llama is trained for improved inductive reasoning performance. |
|
|
|
|
|
### Model Description |
|
|
|
|
|
- **Developed by:** Nafis Sadeq |
|
|
- **Language(s) (NLP):** English |
|
|
- **Finetuned from model:** meta-llama/Meta-Llama-3-8B-Instruct |
|
|
|
|
|
### Model Sources [optional] |
|
|
|
|
|
|
|
|
- **Repository:** https://github.com/NafisSadeq/reasoning-distillation |
|
|
- **Paper:** https://arxiv.org/abs/2504.10647 |
|
|
|
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
|
|
Follow the instructions here: https://github.com/NafisSadeq/reasoning-distillation |
|
|
|
|
|
## Training Details |
|
|
|
|
|
Training details can be found in the paper: https://arxiv.org/abs/2504.10647 |
|
|
|
|
|
## Environmental Impact |
|
|
|
|
|
- **Hardware Type:** 2 × 48 GB Nvidia RTX A6000 GPUs |
|
|
- **Hours used:** 72 hours |
|
|
|
|
|
### Model Architecture and Objective |
|
|
|
|
|
This model has the same architecture as meta-llama/Meta-Llama-3-8B-Instruct |
|
|
|
|
|
### Compute Infrastructure |
|
|
|
|
|
2 × 48 GB Nvidia RTX A6000 GPUs |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you use this model, please cite the following paper. |
|
|
|
|
|
@misc{sadeq2025improvingincontextlearningreasoning, |
|
|
title={Improving In-Context Learning with Reasoning Distillation}, |
|
|
author={Nafis Sadeq and Xin Xu and Zhouhang Xie and Julian McAuley and Byungkyu Kang and Prarit Lamba and Xiang Gao}, |
|
|
year={2025}, |
|
|
eprint={2504.10647}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={cs.CL}, |
|
|
url={https://arxiv.org/abs/2504.10647}, |
|
|
} |
