Update README.md

README.md

@@ -1,3 +1,21 @@
----
-license: mit
----
+---
+license: mit
+---
+# GeLinear
+
+GeLinear is the implementation of [LoLCATs](https://arxiv.org/pdf/2410.10254), but with Gemma 2 model.
+Unlike the original LoLCATs approach that linearizes all attention layers, I focuses on linearizing only the global attention layer, while retaining Gemma 2’s built-in Sliding Window Attention (SWA) for local context (since the time complexity for this already scaled linearly with sequence length). 
+
+To run the model:
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
+model = AutoModelForCausalLM.from_pretrained("jacksonkek/GeLinear",trust_remote_code=True,torch_dtype=torch.bfloat16,device_map="sequential")
+tokenizer = AutoTokenizer.from_pretrained("jacksonkek/GeLinear")
+
+x = "tell me a joke"
+input_text = [{"role":"user","content":x}]
+input_ids = tokenizer.apply_chat_template(input_text, add_generation_prompt=True,return_tensors="pt").to("cuda")
+text_streamer = TextStreamer(tokenizer)
+_ = model.generate(input_ids, streamer = text_streamer, do_sample=False,max_new_tokens = 8192)
+
+```