Disclaimer: The team releasing GIT did not write a model card for this model so this model card has been written by the Hugging Face team.
Model description
GIT is a Transformer decoder conditioned on both CLIP image tokens and text tokens. The model is trained using "teacher forcing" on a lot of (image, text) pairs.
The goal for the model is simply to predict the next text token, giving the image tokens and previous text tokens.
The model has full access to (i.e. a bidirectional attention mask is used for) the image patch tokens, but only has access to the previous text tokens (i.e. a causal attention mask is used for the text tokens) when predicting the next text token.
This allows the model to be used for tasks like:
image and video captioning
visual question answering (VQA) on images and videos
even image classification (by simply conditioning the model on the image and asking it to generate a class for it in text).
Intended uses & limitations
You can use the raw model for image captioning. See the
model hub
to look for
fine-tuned versions on a task that interests you.
We collect 0.8B image-text pairs for pre-training, which include COCO (Lin et al., 2014), Conceptual Captions
(CC3M) (Sharma et al., 2018), SBU (Ordonez et al., 2011), Visual Genome (VG) (Krishna et al., 2016),
Conceptual Captions (CC12M) (Changpinyo et al., 2021), ALT200M (Hu et al., 2021a), and an extra 0.6B
data following a similar collection procedure in Hu et al. (2021a).
=> however this is for the model referred to as "GIT" in the paper, which is not open-sourced.
This checkpoint is "GIT-large", which is a smaller variant of GIT trained on 20 million image-text pairs.
We refer to the original repo regarding details for preprocessing during training.
During validation, one resizes the shorter edge of each image, after which center cropping is performed to a fixed-size resolution. Next, frames are normalized across the RGB channels with the ImageNet mean and standard deviation.
Evaluation results
For evaluation results, we refer readers to the
paper
.
Runs of artificialguybr textcaps-teste2 on huggingface.co
9
Total runs
0
24-hour runs
1
3-day runs
-3
7-day runs
-29
30-day runs
More Information About textcaps-teste2 huggingface.co Model
textcaps-teste2 huggingface.co is an AI model on huggingface.co that provides textcaps-teste2's model effect (), which can be used instantly with this artificialguybr textcaps-teste2 model. huggingface.co supports a free trial of the textcaps-teste2 model, and also provides paid use of the textcaps-teste2. Support call textcaps-teste2 model through api, including Node.js, Python, http.
textcaps-teste2 huggingface.co is an online trial and call api platform, which integrates textcaps-teste2's modeling effects, including api services, and provides a free online trial of textcaps-teste2, you can try textcaps-teste2 online for free by clicking the link below.
artificialguybr textcaps-teste2 online free url in huggingface.co:
textcaps-teste2 is an open source model from GitHub that offers a free installation service, and any user can find textcaps-teste2 on GitHub to install. At the same time, huggingface.co provides the effect of textcaps-teste2 install, users can directly use textcaps-teste2 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.