Update README.md

README.md

@@ -1,3 +1,53 @@
----
-license: mit
----
+convert this to raw readme.md file, it's a model card on huggingface
+
+# Pashto BERT (BERT-Base)
+
+## Model Overview
+This is a monolingual **Pashto BERT (BERT-Base)** model trained on a large **Pashto corpus**. The model is designed to understand and generate text in **Pashto**, making it suitable for various downstream **Natural Language Processing (NLP) tasks**.
+
+## Model Details
+- **Architecture:** BERT-Base (12 layers, 768 hidden size, 12 attention heads, 110M parameters)
+- **Language:** Pashto (ps)
+- **Training Corpus:** A diverse set of Pashto text data, including news articles, books, and web content.
+- **Special Tokens:** `[CLS]`, `[SEP]`, `[PAD]`, `[MASK]`, `[UNK]`
+
+## Intended Use
+This model can be **fine-tuned** for various Pashto-specific NLP tasks, such as:
+- **Sequence Classification:** Sentiment analysis, topic classification, and document categorization.
+- **Sequence Tagging:** Named entity recognition (NER) and part-of-speech (POS) tagging.
+- **Text Generation & Understanding:** Question answering, text summarization, and machine translation.
+
+## How to Use
+This model can be loaded using the `transformers` library from Hugging Face:
+
+```python
+from transformers import AutoModel, AutoTokenizer
+
+model_name = "your-huggingface-username/pashto-bert-base"
+tokenizer = AutoTokenizer.from_pretrained("/kaggle/working/model/")
+model = AutoModel.from_pretrained(model_name)
+
+text = "ستاسو نننۍ ورځ څنګه وه؟"
+tokens = tokenizer(text, return_tensors="pt")
+out = model(**tokens)
+```
+
+## Training Details
+- **Optimization:** AdamW
+- **Sequence Length:** 128
+- **Warmup Steps:** 10,000
+- **Warmup Ratio:** 0.06
+- **Learning Rate:** 1e-4
+- **Weight Decay:** 0.01
+- **Adam Optimizer Parameters:**
+  - **Epsilon:** 1e-8
+  - **Betas:** (0.9, 0.999)
+- **Gradient Accumulation Steps:** 1
+- **Max Gradient Norm:** 1.0
+- **Scheduler:** `linear_schedule_with_warmup`
+
+
+## Limitations & Biases
+- The model may reflect biases present in the training data.
+- Performance on **low-resource or domain-specific tasks** may require additional fine-tuning.
+- It is not trained for **code-switching scenarios** (e.g., mixing Pashto with English or other languages).