|
|
--- |
|
|
language: |
|
|
- en |
|
|
license: apache-2.0 |
|
|
base_model: google/flan-t5-small |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
datasets: |
|
|
- sentence-paraphrases |
|
|
model-index: |
|
|
- name: flan-t5-small-simplifier |
|
|
results: [] |
|
|
--- |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
# flan-t5-small-simplifier |
|
|
|
|
|
For paraphrasing and simplifying English text. |
|
|
|
|
|
Fine-tuned version of [google/flan-t5-small](https://huggingface.co/google/flan-t5-small) on the agentlans/sentence-paraphrases dataset. |
|
|
It achieves the following results on the evaluation set: |
|
|
- Loss: 1.1518 |
|
|
- Num Input Tokens Seen: 32939232 |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
Works best on sentence length texts. |
|
|
|
|
|
``` |
|
|
import torch |
|
|
from transformers import pipeline |
|
|
|
|
|
# Check if GPU is available |
|
|
device = 0 if torch.cuda.is_available() else -1 |
|
|
|
|
|
# Initialize the pipeline |
|
|
model_name = "agentlans/flan-t5-small-simplifier" |
|
|
flan_t5_pipeline = pipeline("text2text-generation", model=model_name, device=device) |
|
|
|
|
|
# Example input |
|
|
input_text = "While navigating the labyrinthine corridors of epistemological uncertainty, the precocious philosopher—whose seminal work on phenomenological interpretation had already garnered significant academic acclaim—paused momentarily to contemplate the intricate interplay between subjective perception and objective reality, ultimately recognizing that the boundaries of human understanding are perpetually fluid and dynamically reconstructed through continuous intellectual discourse and empirical investigation." |
|
|
|
|
|
# Generate output |
|
|
output = flan_t5_pipeline(input_text, max_length=1024) |
|
|
|
|
|
# Print the result |
|
|
print(output[0]["generated_text"]) |
|
|
# The precocious philosopher, who had already been a major academic acclaim for his seminal work on phenomenological interpretation, paused momentarily to contemplate the intricate interplay between subjective perception and objective reality, recognizing that the boundaries of human understanding are perpetually fluid and dynamically reconstructed through continuous intellectual discourse and empirical investigation. |
|
|
``` |
|
|
|
|
|
Limitations: |
|
|
- English only |
|
|
- Doesn't handle mixed language texts well (for example, English with Greek letter words) |
|
|
- Might not be able to simplify some texts |
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
agentlans/sentence-paraphrases |
|
|
|
|
|
This dataset is a curated collection of sentence-length paraphrases derived from two primary sources: |
|
|
|
|
|
humarin/chatgpt-paraphrases |
|
|
xwjzds/paraphrase_collections. |
|
|
|
|
|
Dataset Details |
|
|
Dataset Description |
|
|
|
|
|
The dataset is structured to provide pairs of sentences from an original text and its paraphrase(s). For each entry: |
|
|
|
|
|
The "text" field contains the least readable paraphrase. |
|
|
The "paraphrase" field contains the most readable paraphrase. |
|
|
|
|
|
Readability was assessed using the agentlans/deberta-v3-xsmall-zyda-2-readability model. |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 5e-05 |
|
|
- train_batch_size: 8 |
|
|
- eval_batch_size: 8 |
|
|
- seed: 42 |
|
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
|
- lr_scheduler_type: linear |
|
|
- num_epochs: 2.0 |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen | |
|
|
|:-------------:|:------:|:-----:|:---------------:|:-----------------:| |
|
|
| 1.4423 | 0.2224 | 10000 | 1.2431 | 3655312 | |
|
|
| 1.3884 | 0.4448 | 20000 | 1.2093 | 7331520 | |
|
|
| 1.3782 | 0.6673 | 30000 | 1.1859 | 10990432 | |
|
|
| 1.3595 | 0.8897 | 40000 | 1.1787 | 14653328 | |
|
|
| 1.3059 | 1.1121 | 50000 | 1.1665 | 18326104 | |
|
|
| 1.3298 | 1.3345 | 60000 | 1.1589 | 21991016 | |
|
|
| 1.2994 | 1.5569 | 70000 | 1.1562 | 25656600 | |
|
|
| 1.2952 | 1.7794 | 80000 | 1.1518 | 29314808 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.43.3 |
|
|
- Pytorch 2.3.0+cu121 |
|
|
- Datasets 3.2.0 |
|
|
- Tokenizers 0.19.1 |
|
|
|
