README.md

---
base_model: google/flan-t5-large
library_name: peft
tags:
- text-generation
- question-answering
- bias-mitigation
- flan-t5
- lora
- peft
- disability-rights
- accessibility
- social-impact
---

# Flan-T5-Large LoRA Adapter for Disability Q&A and Mitigating Disability Biases

## Model Description

This repository contains a LoRA (Low-Rank Adaptation) adapter fine-tuned on the `google/flan-t5-large` base model. The adapter is specifically trained for **improving question-answering capabilities related to disability information and actively reducing harmful biases and stereotypes concerning people with disabilities in generated text.**

This model leverages the PEFT (Parameter-Efficient Fine-Tuning) library to efficiently adapt the large Flan-T5 model to this specialized domain without requiring full model retraining, making it more resource-efficient and deployable.

-   **Developed by:** omark807
-   **Finetuned from model:** `google/flan-t5-large`
-   **Model type:** Adapter (LoRA) for Sequence-to-Sequence Language Model
-   **Language(s) (NLP):** English
-   **License:** GPL

### Base Model Details (`google/flan-t5-large`)

Flan-T5 is an instruction-tuned variant of the T5 text-to-text transformer model. It has been fine-tuned on a collection of datasets expressed as natural language instructions. The "large" version has approximately 770 million parameters. This adapter builds upon its strong instruction-following capabilities.

* **Original Model Card:** [https://huggingface.co/google/flan-t5-large](https://huggingface.co/google/flan-t5-large)

## Uses

### Direct Use

This adapter is intended to be loaded alongside the `google/flan-t5-large` model using the PEFT library. It can then be used for:

* **Answering questions** related to various aspects of disability, accessibility, disability rights, legislation, and common challenges.
* **Generating responses** that are more inclusive, respectful, and free from common disability biases and stereotypes.
* **Providing information** in a neutral and empathetic tone when discussing disability-related topics.

**Example Inference for Q&A:**
```python
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel, PeftConfig
import torch

# Load the base model
model_name = "google/flan-t5-large"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

# Load your adapter
# Replace "your-huggingface-username/your-repo-name" with your actual model ID
adapter_model_id = "[your-huggingface-username]/[your-repo-name]" 
model = PeftModel.from_pretrained(model, adapter_model_id)
model.eval() # Set model to evaluation mode

# Example inference for Q&A
# Input: "What is the Americans with Disabilities Act (ADA)?"
# Expected Output: A concise explanation of the ADA.
input_text_qa = "question: What is the Americans with Disabilities Act (ADA)?"
input_ids_qa = tokenizer(input_text_qa, return_tensors="pt").input_ids

with torch.no_grad():
    outputs_qa = model.generate(input_ids_qa, max_new_tokens=100, num_beams=5, early_stopping=True)

decoded_output_qa = tokenizer.decode(outputs_qa[0], skip_special_tokens=True)
print(f"Input (Q&A): {input_text_qa}")
print(f"Output (Q&A): {decoded_output_qa}")

# Example inference for Bias Mitigation/Instruction Following
# Input: "Rewrite the following sentence to remove any ableist language: 'He was confined to a wheelchair.'"
# Expected Output: "He used a wheelchair." or similar respectful phrasing.
input_text_bias = "instruction: Rewrite the following sentence to remove any ableist language: 'He was confined to a wheelchair.'"
input_ids_bias = tokenizer(input_text_bias, return_tensors="pt").input_ids

with torch.no_grad():
    outputs_bias = model.generate(input_ids_bias, max_new_tokens=50, num_beams=5, early_stopping=True)

decoded_output_bias = tokenizer.decode(outputs_bias[0], skip_special_tokens=True)
print(f"Input (Bias Mitigation): {input_text_bias}")
print(f"Output (Bias Mitigation): {decoded_output_bias}")