Total runs:
418
Run Growth:
60
Growth Rate:
13.61%
Sponsored by
i10X
keras-io / shiftvit
Model's Last Updated: July 05 2024
image-classification
Introduction of shiftvit
Model Details of shiftvit
Model description
ShiftViT is a variation of the Vision Transformer (ViT) where the attention operation has been replaced with a shifting operation.
ShiftViT model was proposed as part of the paper When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanism . Vision Transformers have lately become very popular for computer vision problems and a lot researchers attribute their success to the attention layers. The authors of the ShiftViT paper have tried to show via the ShiftViT model that even without the attention operation, ViTs can reach SoTA results.
Model Architecture
The architecture for ShiftViT is inspired by the paper Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Here the authors propose a modular architecture with 4 stages. Each stage works on its own spatial size, creating a hierarchical architecture.
|
|---|
| Figure 1: The entire architecutre of ShiftViT. |
| Source |
Each stage in the ShiftViT architecture comprises of a Shift Block as shown in Fig 2.
|
|---|
| Figure 2: From the Model to a Shift Block. |
The Shift Block as shown in Fig. 2, comprises of the following:
- Shift Operation
- Linear Normalization
- MLP Layer: stack of densely-connected layers
How shift operation works:
- Split the channels
- Select each of the first four spilts and shift and pad them in the respective directions.
- After shifting and padding, we concatenate the channel back.
|
|---|
| Figure 3: The TensorFlow style shifting |
Intended uses
This ShiftViT model is trained to be used for image classification task.
However, the ShiftViT architecture can be used for a variety of visual recognition tasks. The authors of the ShiftViT paper tested the model on the following tasks:
- Image Classification on ImageNet-1k dataset
- Object Detection on COCO dataset
- Semantic Segmentation on ADE20k dataset
Training and evaluation data
The dataset used for training the model is CIFAR-10. The CIFAR-10 dataset is a popular dataset used for image classification. It contains images belonging to the following 10 classes:
| Classes |
|---|
| airplane |
| automobile |
| bird |
| cat |
| deer |
| dog |
| frog |
| horse |
| ship |
| truck |
No. of samples used for training and evaluation are:
- Training samples: 40000
- Validation samples: 10000
- Testing samples: 10000
Training procedure
- Data Preparation:
- Data Augmentation: The augmentation steps used for data preparation include: rescaling, resizing, cropping and horizontal flipping.
- Building the ShiftViT Model:
- The steps for constructing the ShiftViT model have been covered extensively in this Keras example
- Model Training:
The model is then trained using the following config:
| Training Config | Value |
|---|---|
| Optimizer | Adam |
| Loss Function | sparse_categorical_crossentropy |
| Metric | Accuracy |
| Epochs | 5 |
- Model Testing:
The model is tested on the test data post training achieving an accuracy of ~90%.
Training hyperparameters
The following hyperparameters were used during training:
| Hyperparameters | Value |
|---|---|
| name | AdamW |
| learning_rate.class_name | WarmUpCosine |
| learning_rate.config.lr_start | 1e-05 |
| learning_rate.config.lr_max | 0.001 |
| learning_rate.config.total_steps | 312 |
| learning_rate.config.warmup_steps | 46 |
| decay | 0.0 |
| beta_1 | 0.8999999761581421 |
| beta_2 | 0.9990000128746033 |
| epsilon | 1e-07 |
| amsgrad | False |
| weight_decay | 9.999999747378752e-05 |
| exclude_from_weight_decay | None |
| training_precision | float32 |
Model Plot
View Model Plot
Credits:
- HF Contribution: Shivalika Singh
- Full credits to original Keras example by Aritra Roy Gosthipaty and Ritwik Raha
- Check out the demo space here
Runs of keras-io shiftvit on huggingface.co
10
Total runs
0
24-hour runs
0
3-day runs
-7
7-day runs
-2
30-day runs
More Information About shiftvit huggingface.co Model
Url of shiftvit
shiftvit huggingface.co Url
Provider of shiftvit huggingface.co
Other API from keras-io
Updated:July 09 2024
Total runs:
214
Run Growth:
115
Growth Rate:
54.25%
Total runs:
155
Run Growth:
-122
Growth Rate:
-78.71%
Total runs:
81
Run Growth:
74
Growth Rate:
90.24%
Total runs:
79
Run Growth:
56
Growth Rate:
70.89%
Total runs:
49
Run Growth:
8
Growth Rate:
16.33%
Total runs:
38
Run Growth:
13
Growth Rate:
33.33%
Total runs:
34
Run Growth:
-2
Growth Rate:
-6.25%
Total runs:
34
Run Growth:
-7
Growth Rate:
-20.00%
Total runs:
33
Run Growth:
-39
Growth Rate:
-121.88%
Total runs:
29
Run Growth:
-1
Growth Rate:
-3.57%
Total runs:
28
Run Growth:
21
Growth Rate:
75.00%
Total runs:
23
Run Growth:
-5
Growth Rate:
-20.00%
Total runs:
22
Run Growth:
-7
Growth Rate:
-33.33%
Total runs:
21
Run Growth:
-15
Growth Rate:
-68.18%
Total runs:
21
Run Growth:
-18
Growth Rate:
-85.71%
Total runs:
20
Run Growth:
-11
Growth Rate:
-61.11%
Total runs:
18
Run Growth:
-21
Growth Rate:
-123.53%
Total runs:
18
Run Growth:
-41
Growth Rate:
-292.86%
Total runs:
15
Run Growth:
8
Growth Rate:
53.33%
Total runs:
15
Run Growth:
-5
Growth Rate:
-33.33%
Total runs:
14
Run Growth:
-12
Growth Rate:
-120.00%
Total runs:
14
Run Growth:
-4
Growth Rate:
-23.53%
Total runs:
14
Run Growth:
-9
Growth Rate:
-64.29%
Total runs:
13
Run Growth:
-6
Growth Rate:
-46.15%
Total runs:
12
Run Growth:
5
Growth Rate:
38.46%
Total runs:
12
Run Growth:
-4
Growth Rate:
-36.36%
Total runs:
11
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
2
Growth Rate:
16.67%
Total runs:
9
Run Growth:
-25
Growth Rate:
-357.14%
Total runs:
8
Run Growth:
5
Growth Rate:
55.56%
Total runs:
8
Run Growth:
-12
Growth Rate:
-133.33%
Total runs:
8
Run Growth:
-5
Growth Rate:
-62.50%
Total runs:
8
Run Growth:
1
Growth Rate:
11.11%
Total runs:
8
Run Growth:
-1
Growth Rate:
-12.50%
Total runs:
8
Run Growth:
1
Growth Rate:
10.00%
Total runs:
7
Run Growth:
-42
Growth Rate:
-600.00%
Total runs:
7
Run Growth:
-9
Growth Rate:
-90.00%
Total runs:
7
Run Growth:
-3
Growth Rate:
-42.86%
Total runs:
7
Run Growth:
6
Growth Rate:
85.71%
Total runs:
7
Run Growth:
3
Growth Rate:
42.86%
Total runs:
7
Run Growth:
1
Growth Rate:
14.29%
Total runs:
6
Run Growth:
-12
Growth Rate:
-200.00%
Total runs:
6
Run Growth:
-13
Growth Rate:
-216.67%
Total runs:
6
Run Growth:
2
Growth Rate:
28.57%
Total runs:
6
Run Growth:
-7
Growth Rate:
-77.78%
Total runs:
5
Run Growth:
-4
Growth Rate:
-80.00%
Total runs:
5
Run Growth:
1
Growth Rate:
20.00%
Total runs:
4
Run Growth:
-5
Growth Rate:
-100.00%
Total runs:
4
Run Growth:
-2
Growth Rate:
-50.00%
Total runs:
4
Run Growth:
-9
Growth Rate:
-225.00%
Total runs:
3
Run Growth:
1
Growth Rate:
33.33%
Total runs:
3
Run Growth:
-5
Growth Rate:
-166.67%
Total runs:
3
Run Growth:
0
Growth Rate:
0.00%
Total runs:
2
Run Growth:
-1
Growth Rate:
-50.00%
Total runs:
2
Run Growth:
2
Growth Rate:
66.67%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%