Add link to project page and correct pipeline tag
Browse filesThis PR makes sure the model card is linked to the project page and the pipeline tag is set to any-to-any.
README.md
CHANGED
@@ -1,8 +1,8 @@
|
|
1 |
---
|
2 |
-
license: apache-2.0
|
3 |
-
library_name: transformers
|
4 |
base_model: OpenGVLab/InternVL2-4B
|
5 |
-
|
|
|
|
|
6 |
---
|
7 |
|
8 |
# OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis
|
@@ -137,9 +137,15 @@ tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True, use_fast
|
|
137 |
pixel_values = load_image('./web_dfacd48d-d2c2-492f-b94c-41e6a34ea99f.png', max_num=6).to(torch.bfloat16).cuda()
|
138 |
generation_config = dict(max_new_tokens=1024, do_sample=True)
|
139 |
|
140 |
-
question = "<image
|
|
|
|
|
|
|
|
|
|
|
141 |
response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=None, return_history=True)
|
142 |
-
print(f'User: {question}
|
|
|
143 |
```
|
144 |
|
145 |
|
|
|
1 |
---
|
|
|
|
|
2 |
base_model: OpenGVLab/InternVL2-4B
|
3 |
+
library_name: transformers
|
4 |
+
license: apache-2.0
|
5 |
+
pipeline_tag: any-to-any
|
6 |
---
|
7 |
|
8 |
# OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis
|
|
|
137 |
pixel_values = load_image('./web_dfacd48d-d2c2-492f-b94c-41e6a34ea99f.png', max_num=6).to(torch.bfloat16).cuda()
|
138 |
generation_config = dict(max_new_tokens=1024, do_sample=True)
|
139 |
|
140 |
+
question = "<image>
|
141 |
+
You are a GUI task expert, I will provide you with a high-level instruction, an action history, a screenshot with its corresponding accessibility tree.
|
142 |
+
High-level instruction: {high_level_instruction}
|
143 |
+
Action history: {action_history}
|
144 |
+
Accessibility tree: {a11y_tree}
|
145 |
+
Please generate the low-level thought and action for the next step."
|
146 |
response, history = model.chat(tokenizer, pixel_values, question, generation_config, history=None, return_history=True)
|
147 |
+
print(f'User: {question}
|
148 |
+
Assistant: {response}')
|
149 |
```
|
150 |
|
151 |
|