Total runs:
91
Run Growth:
0
Growth Rate:
0.00%
Sponsored by
YouArt
nlpconnect / vit-gpt2-image-captioning
Model's Last Updated: February 27 2023
image-to-text
Introduction of vit-gpt2-image-captioning
Model Details of vit-gpt2-image-captioning
nlpconnect/vit-gpt2-image-captioning
This is an image captioning model trained by @ydshieh in flax this is pytorch version of this .
The Illustrated Image Captioning using transformers
Sample running code
from transformers import VisionEncoderDecoderModel, ViTImageProcessor, AutoTokenizer
import torch
from PIL import Image
model = VisionEncoderDecoderModel.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
feature_extractor = ViTImageProcessor.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
tokenizer = AutoTokenizer.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
max_length = 16
num_beams = 4
gen_kwargs = {"max_length": max_length, "num_beams": num_beams}
def predict_step(image_paths):
images = []
for image_path in image_paths:
i_image = Image.open(image_path)
if i_image.mode != "RGB":
i_image = i_image.convert(mode="RGB")
images.append(i_image)
pixel_values = feature_extractor(images=images, return_tensors="pt").pixel_values
pixel_values = pixel_values.to(device)
output_ids = model.generate(pixel_values, **gen_kwargs)
preds = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
preds = [pred.strip() for pred in preds]
return preds
predict_step(['doctor.e16ba4e4.jpg']) # ['a woman in a hospital bed with a woman in a hospital bed']
Sample running code using transformers pipeline
from transformers import pipeline
image_to_text = pipeline("image-to-text", model="nlpconnect/vit-gpt2-image-captioning")
image_to_text("https://ankur3107.github.io/assets/images/image-captioning-example.png")
# [{'generated_text': 'a soccer game with a player jumping to catch the ball '}]
Contact for any help
Runs of nlpconnect vit-gpt2-image-captioning on huggingface.co
85.2K
Total runs
0
24-hour runs
-1.4K
3-day runs
-12.2K
7-day runs
-25.6K
30-day runs
More Information About vit-gpt2-image-captioning huggingface.co Model
Url of vit-gpt2-image-captioning
vit-gpt2-image-captioning huggingface.co Url
Provider of vit-gpt2-image-captioning huggingface.co
Other API from nlpconnect
Updated:December 06 2022
Total runs:
79
Run Growth:
62
Growth Rate:
78.48%
Total runs:
27
Run Growth:
-20
Growth Rate:
-83.33%
Total runs:
26
Run Growth:
17
Growth Rate:
65.38%
Total runs:
25
Run Growth:
16
Growth Rate:
64.00%
Total runs:
20
Run Growth:
-18
Growth Rate:
-100.00%
Total runs:
14
Run Growth:
9
Growth Rate:
64.29%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%