Hugging Face's logo

google
/
gemma-2b

Text Generation
Transformers
Safetensors
GGUF
gemma
text-generation-inference
Model card Files Community
81

What is the context size for Gemma? I get error when asking for it in the config file e.g., AttributeError("'GemmaConfig' object has no attribute 'context_length'")

#32
by brando - opened
Discussion
brando

What is the context size for Gemma? I get error when asking for it in the config file e.g., AttributeError("'GemmaConfig' object has no attribute 'context_length'")

ybelkada
•
edited Mar 13, 2024

Hi ! I think it should be 8192: https://huggingface.co/google/gemma-2b/blob/main/config.json#L14

ybelkada

This has been confirmed by the author here: https://huggingface.co/google/gemma-7b-it/discussions/73#65e9678c0cda621164a95bad

lkv
 Google org

Hi @brando , The 2B model actually uses MQA (just 1 KV head), whereas the 7B uses MHA. Both models have the same sequence/context length of 8192, as specified in the technical report. please find this report for reference. Thank you.

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment