Hugging Face's logo

KonstantinosKK
/
lavida-llada-v1.0-instruct-hf-transformers

Image-Text-to-Text
Transformers
Safetensors
llava_llada
text-generation
conversational
custom_code
Model card Files
xet
Community
1
lavida-llada-v1.0-instruct-hf-transformers / README.md
KonstantinosKK's picture
KonstantinosKK
Improve model card metadata (#1)
6abbe20 verified
preview code
|
raw
history blame contribute delete
1.1 kB
metadata
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text

LaViDa-LLaDa v1.0 Instruct (Transformers-Compatible)

[Github][Paper] [Arxiv] [Checkpoints] [Data] [Website]

This is a transformers-compatible version of the LaViDa-LLaDa checkpoint. It allows direct loading via Huggingface transformers APIs for easier inference and integration.

Usage 

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained('./lavida-llada-v1.0-instruct/')
model = AutoModelForCausalLM.from_pretrained('./lavida-llada-v1.0-instruct/', torch_dtype=torch.bfloat16)
image_processor = model.get_vision_tower().image_processor

model.resize_token_embeddings(len(tokenizer))
model.tie_weights()

License

Apache 2.0