File size: 2,217 Bytes
6eaddc2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
---
library_name: peft
base_model: Qwen/Qwen2.5-Coder-3B-Instruct-AWQ
tags:
- knowledge-distillation
- code-generation
- qwen
- lora
- distilled
license: apache-2.0
---
# Qwen2.5-Coder-3B Distilled Model
This is a **knowledge-distilled** version of Qwen2.5-Coder-3B-Instruct-AWQ, trained using knowledge distillation from Qwen2.5-Coder-7B-Instruct-AWQ.
## Model Details
- **Base Model**: Qwen/Qwen2.5-Coder-3B-Instruct-AWQ
- **Teacher Model**: Qwen/Qwen2.5-Coder-7B-Instruct-AWQ
- **Training Method**: Knowledge Distillation with LoRA
- **Best Validation Loss**: 1.9286
- **Training Time**: ~5 minutes
- **Parameters Trained**: 14.9M (4.59% of base model)
## Training Configuration
- **Temperature**: 2.0 (optimal)
- **Alpha**: 0.95 (95% distillation weight)
- **LoRA Rank**: 8
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-Coder-3B-Instruct-AWQ",
torch_dtype=torch.float16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-3B-Instruct-AWQ")
# Load distilled adapter
model = PeftModel.from_pretrained(base_model, "Vinitha2004/qwen-coder-1.5B-Instruct-AWQ-t2")
# Generate code
input_text = "Original Code:\ndef add(a, b):\n return a + b\n\nUpdate Snippet:\n// ... existing code ...\ndef add(a: int, b: int) -> int:\n// ... existing code ...\n\nUpdated Code:\n"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
```
## Performance
This distilled model retains the knowledge from the 7B teacher model while being significantly more efficient:
- **Faster inference** (3B vs 7B parameters)
- **Lower memory usage**
- **Maintained code generation quality**
## Training Dataset
Trained on 5000 code editing examples from custom dataset.
## Files
- `adapter_config.json`: LoRA configuration
- `adapter_model.safetensors`: Trained LoRA weights (59MB)
- Other standard tokenizer files
|