Add pipeline tag and library name

README.md

@@ -1,9 +1,13 @@
 ---
 license: mit
+library_name: transformers
+pipeline_tag: question-answering
 ---
 
 # Cuckoo 🐦 [[Github]](https://github.com/KomeijiForce/Cuckoo)
 
+The Cuckoo family of models are extractive question answering models as described in the paper [Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM's Nest](https://hf.co/papers/2502.11275).
+
 Cuckoo is a small (300M) information extraction (IE) model that imitates the next token prediction paradigm of large language models. Instead of retrieving from the vocabulary, Cuckoo predicts the next tokens by tagging them in the given input context as shown below:
 
 ![cuckoo](https://github.com/user-attachments/assets/d000f275-82a7-4939-aca8-341c61a774dc)
@@ -40,9 +44,16 @@ Begin your journey with Cuckoo to experience unimaginable adaptation efficiency
 | └─ Only Post-train   | 72.80     | 56.10     | 66.02          | 67.10     | 65.51 | 68.66     | 69.75 | 69.21 | 77.05 | 62.39    | 54.80 | 64.75 |
 | Rainbow Cuckoo 🌈🐦🛠️  | 79.94     | 58.39     | 70.30          | 67.00     | **68.91** | 70.47     | 76.05 | **73.26** | 86.57 | 69.41    | 64.64 | **73.54** |
 
+*(Super Rainbow Cuckoo 🦸🌈🐦🛠️  uses training sets except CoNLL2004 and ADE to boost its performance)*
+
+|                      | CoNLL2003 | BioNLP2004 | MIT-Restaurant | MIT-Movie | Avg.  | CoNLL2004 | ADE   | Avg.  | SQuAD | SQuAD-V2 | DROP  | Avg.  |
+|----------------------|-----------|-----------|----------------|-----------|-------|-----------|-------|-------|-------|----------|-------|-------|
+| Super Rainbow Cuckoo 🦸🌈🐦🛠️           | 88.38     | 68.33     | 76.79          | 69.39     | **75.22** | 72.96     | 80.06  | **76.51** | 89.54 | 74.52    | 74.89  | **79.65** |
+
+
 ## Quick Experience with Cuckoo in Next Tokens Extraction ⚡
 
-We recommend using the strongest Super Rainbow Cuckoo 🦸🌈🐦🛠️ for zero-shot extraction.
+We recommend using the strongest Super Rainbow Cuckoo 🦸🌈🐦🛠️ for zero-shot extraction. You can directly run the cases below in ```case_next_tokens_extraction.py```.
 
 1️⃣ First load the model and the tokenizers
 
@@ -97,23 +108,23 @@ Case 1: Basic entity and relation understanding
 text = "Tom and Jack went to their trip in Paris."
 
 for question in [
-    "What are the people mentioned here?",
+    "What is the person mentioned here?",
     "What is the city mentioned here?",
     "Who goes with Tom together?",
     "What do Tom and Jack go to Paris for?",
-    "Which city does George live in?",
+    "Where does George live in?",
 ]:
-    text = f"User:\n\n{text}\n\nQuestion: {question}\n\nAssistant:"
-    predictions = next_tokens_extraction(text)
+    prompt = f"User:\n\n{text}\n\nQuestion: {question}\n\nAssistant:"
+    predictions = next_tokens_extraction(prompt)
     print(question, predictions)
 ```
 You will get things like,
 ```
-What are the people mentioned here? ['Tom', 'Jack']
+What is the person mentioned here? ['Tom', 'Jack']
 What is the city mentioned here? ['Paris']
 Who goes with Tom together? ['Jack']
 What do Tom and Jack go to Paris for? ['trip']
-Which city does George live in? []
+Where does George live in? []
 ```
 where [] indicates Cuckoo thinks there to be no next tokens for extraction.
 
@@ -143,8 +154,7 @@ When did Beethoven's middle period showed an individual development? ['1802']
 Case 3: Knowledge quiz
 
 ```python
-for obj in ["grass", "sea", "fire", "night"]:
-    text = f"User:\n\nChoices:\nred\nblue\ngreen.\n\nQuestion: What is the color of the {obj}?\n\nAssistant:\n\nAnswer:"
+for obj in ["grass", "sea", "fire", "night"]:\n    text = f"User:\\n\\nChoices:\\nred\\nblue\\ngreen.\\n\\nQuestion: What is the color of the {obj}?\\n\\nAssistant:\\n\\nAnswer:"
     predictions = next_tokens_extraction(text)
     print(obj, predictions)
 ```
@@ -155,4 +165,79 @@ sea ['blue']
 fire ['red']
 night []
 ```
-which shows Cuckoo is not extracting any plausible spans but has the knowledge to understand the context.
+which shows Cuckoo is not extracting any plausible spans but has the knowledge to understand the context.
+
+## Few-shot Adaptation 🎯
+
+Cuckoo 🐦 is an expert in few-shot adaptation to your own tasks, taking CoNLL2003 as an example, run ```bash run_downstream.sh conll2003.5shot KomeijiForce/Cuckoo-C4-Rainbow```, you will get a fine-tuned model in ```models/cuckoo-conll2003.5shot```. Then you can benchmark the model with the script ```python eval_conll2003.py```, which will show you an F1 performance of around 80.
+
+You can also train the adaptation to machine reading comprehension (SQuAD), run ```bash run_downstream.sh squad.32shot KomeijiForce/Cuckoo-C4-Rainbow```, you will get a fine-tuned model in ```models/cuckoo-squad.32shot```. Then you can benchmark the model with the script ```python eval_squad.py```, which will show you an F1 performance of around 88.
+
+For fine-tuning your own task, you need to create a Jsonlines file, each line contains {"words": [...], "ner": [...]}, For example:
+
+```json
+{"words": ["I", "am", "John", "Smith", ".", "Person", ":"], "ner": ["O", "O", "B", "I", "O", "O", "O"]}
+```
+
+<img src="https://github.com/user-attachments/assets/ef177466-d915-46d2-9201-5e672bb6ec23" style="width: 40%;" />
+
+which indicates "John Smith" to be predicted as the next tokens.
+
+You can refer to some prompts shown below for beginning:
+
+| **Type**            | **User Input**                                                                                      | **Assistant Response**                              |
+|---------------------|----------------------------------------------------------------------------------------------------|----------------------------------------------------|
+| Entity              | **User:** [Context] Question: What is the [Label] mentioned?                                        | **Assistant:** Answer: The [Label] is             |
+| Relation (Kill)     | **User:** [Context] Question: Who does [Entity] kill?                                               | **Assistant:** Answer: [Entity] kills             |
+| Relation (Live)     | **User:** [Context] Question: Where does [Entity] live in?                                          | **Assistant:** Answer: [Entity] lives in         |
+| Relation (Work)     | **User:** [Context] Question: Who does [Entity] work for?                                           | **Assistant:** Answer: [Entity] works for         |
+| Relation (Located)  | **User:** [Context] Question: Where is [Entity] located in?                                         | **Assistant:** Answer: [Entity] is located in     |
+| Relation (Based)    | **User:** [Context] Question: Where is [Entity] based in?                                           | **Assistant:** Answer: [Entity] is based in       |
+| Relation (Adverse)  | **User:** [Context] Question: What is the adverse effect of [Entity]?                               | **Assistant:** Answer: The adverse effect of [Entity] is  |
+| Query               | **User:** [Context] Question: [Question]                                                           | **Assistant:** Answer:                            |
+| Instruction (Entity)| **User:** [Context] Question: What is the [Label] mentioned? ([Instruction])                        | **Assistant:** Answer: The [Label] is             |
+| Instruction (Query) | **User:** [Context] Question: [Question] ([Instruction])                                            | **Assistant:** Answer:                           |
+
+After building your own downstream dataset, save it into ```my_downstream.json```, and then run the command ```bash run_downstream.sh my_downstream KomeijiForce/Cuckoo-C4-Rainbow```. You will find an adapted Cuckoo in ```models/cuckoo-my_downstream```.
+
+## Fly your own Cuckoo 🪽
+
+We include the script to transform texts to NTE instances in the file ```nte_data_collection.py```, which takes C4 as an example, the converted results can be checked in ```cuckoo.c4.example.json```. The script is designed to be easily adapted to other resources like entity, query, and questions and you can modify your own data to NTE to fly your own Cuckoo! Run the ```run_cuckoo.sh``` script to try an example pre-training.
+
+```bash
+python run_ner.py \
+  --model_name_or_path roberta-large \
+  --train_file cuckoo.c4.example.json \
+  --output_dir models/cuckoo-c4-example \
+  --per_device_train_batch_size 4\
+  --gradient_accumulation_steps 16\
+  --num_train_epochs 1\
+  --save_steps 1000\
+  --learning_rate 0.00001\
+  --do_train \
+  --overwrite_output_dir
+```
+
+You will get an example Cuckoo model in ```models/cuckoo-c4-example```, it might not perform well if you pre-train with too little data. You may adjust the hyperparameters inside ```nte_data_collection.py``` or modify the conversion for your own resources to enable better pre-training performance.
+
+## 🐾 Citation
+
+```
+@article{DBLP:journals/corr/abs-2502-11275,
+  author       = {Letian Peng and
+                  Zilong Wang and
+                  Feng Yao and
+                  Jingbo Shang},
+  title        = {Cuckoo: An {IE} Free Rider Hatched by Massive Nutrition in {LLM}'s Nest},
+  journal      = {CoRR},
+  volume       = {abs/2502.11275},
+  year         = {2025},
+  url          = {https://doi.org/10.48550/arXiv.2502.11275},
+  doi          = {10.48550/arXiv.2502.11275},
+  eprinttype   = {arXiv},
+  eprint       = {2502.11275},
+  timestamp    = {Mon, 17 Feb 2025 19:32:20 +0000},
+  biburl       = {https://dblp.org/rec/journals/corr/abs-2502-11275.bib},
+  bibsource    = {dblp computer science bibliography, https://dblp.org}
+}
+```