Pocket-Llama2-3.2-3B-Instruct

Pocket-Llama2-3.2-3B-Instruct is based on the Llama 3.2 architecture, designed as a lightweight and efficient general-purpose chat assistant. Optimized for fast inference while maintaining strong problem-solving, mathematical reasoning, and scientific capabilities. This model is fine-tuned for enhanced structured reasoning, minimal token wastage, and high-quality technical responses.

Key Improvements

1. Optimized for General Purpose Chat: Excels in a wide range of topics, including casual conversation, technical discussions, and knowledge-based queries.
2. Strong Math & Science Capabilities: Provides accurate and structured explanations for mathematical and scientific problems.
3. Compact yet Powerful: Maintains strong problem-solving capabilities within a smaller 3B parameter architecture, ensuring accessibility on resource-limited devices.
4. Advanced Reasoning Capabilities: Excels in algorithmic problem-solving, structured technical explanations, and logical analysis.
5. Efficient Memory Utilization: Reduces computational overhead while maintaining high-quality outputs.
6. Focused Output Generation: Avoids unnecessary token generation, ensuring concise and relevant responses.

Quickstart with transformers

Here is a code snippet to load the tokenizer and model using apply_chat_template for structured input formatting:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "prithivMLmods/Pocket-Llama2-3.2-3B-Instruct"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Explain the theory of relativity in simple terms."
messages = [
    {"role": "system", "content": "You are an advanced assistant specialized in science and mathematics."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=6090
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

Intended Use

1. General Chat & Knowledge-Based Queries:
   Engages in informative and casual discussions on a wide range of topics.
2. Mathematics & Science Problem Solving:
   Provides accurate calculations and structured explanations for complex problems.
3. Technical Documentation & Explanation:
   Assists in generating well-structured documentation for APIs, scientific concepts, and coding principles.
4. Debugging Assistance:
   Helps identify and correct errors in code snippets.
5. Educational Support:
   Simplifies complex topics for students and learners with clear explanations.
6. Structured Data Processing:
   Generates structured outputs like JSON, XML, and tables for data science applications.

Limitations

1. Hardware Constraints:
   Although lighter than larger models, still requires a moderately powerful GPU or TPU for optimal performance.
2. Potential Bias in Responses:
   Outputs may reflect biases present in training data.
3. Limited Creativity:
   May generate variable results in non-technical, creative tasks.
4. No Real-Time Awareness:
   Lacks access to real-world events beyond its training cutoff.
5. Error Propagation in Long Responses:
   Minor mistakes in early outputs may affect overall coherence in lengthy responses.
6. Prompt Sensitivity:
   The effectiveness of responses depends on well-structured prompts.