---
library_name: peft
base_model: Qwen/Qwen2.5-Coder-3B-Instruct-AWQ
tags:
- knowledge-distillation
- code-generation
- qwen
- lora
- distilled
license: apache-2.0
---

# Qwen2.5-Coder-3B Distilled Model

This is a **knowledge-distilled** version of Qwen2.5-Coder-3B-Instruct-AWQ, trained using knowledge distillation from Qwen2.5-Coder-7B-Instruct-AWQ.

## Model Details

- **Base Model**: Qwen/Qwen2.5-Coder-3B-Instruct-AWQ
- **Teacher Model**: Qwen/Qwen2.5-Coder-7B-Instruct-AWQ
- **Training Method**: Knowledge Distillation with LoRA
- **Best Validation Loss**: 1.9286
- **Training Time**: ~5 minutes
- **Parameters Trained**: 14.9M (4.59% of base model)

## Training Configuration

- **Temperature**: 2.0 (optimal)
- **Alpha**: 0.95 (95% distillation weight)
- **LoRA Rank**: 8
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-Coder-3B-Instruct-AWQ",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-3B-Instruct-AWQ")

# Load distilled adapter
model = PeftModel.from_pretrained(base_model, "Vinitha2004/qwen-coder-1.5B-Instruct-AWQ-t2")

# Generate code
input_text = "Original Code:\ndef add(a, b):\n    return a + b\n\nUpdate Snippet:\n// ... existing code ...\ndef add(a: int, b: int) -> int:\n// ... existing code ...\n\nUpdated Code:\n"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
```

## Performance

This distilled model retains the knowledge from the 7B teacher model while being significantly more efficient:
- **Faster inference** (3B vs 7B parameters)
- **Lower memory usage**
- **Maintained code generation quality**

## Training Dataset

Trained on 5000 code editing examples from custom dataset.

## Files

- `adapter_config.json`: LoRA configuration
- `adapter_model.safetensors`: Trained LoRA weights (59MB)
- Other standard tokenizer files