Total runs:
121
Run Growth:
-47
Growth Rate:
-40.87%
Sponsored by
PoYo.AI
keras-io / learning_to_tokenize_in_ViT
Model's Last Updated: July 05 2024
image-classification
Introduction of learning_to_tokenize_in_ViT
Model Details of learning_to_tokenize_in_ViT
Tensorflow Keras implementation of : Learning to tokenize in Vision Transformers
The full credit goes to: Aritra Roy Gosthipaty , Sayak Paul
Short description:
ViT and other Transformer based architectures break down images into patches. As we increase the resolution of the images, the number of patches increases as well. To tackle this, Ryoo et al. introduced a new module called TokenLearner which can help reduce the number of patches used. The full paper can be found here
Model and Dataset used
The Dataset used here is CIFAR-10. The model used here is a mini ViT model with the TokenLearner module.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
| Hyperparameters | Value |
|---|---|
| name | AdamW |
| learning_rate | 0.0010000000474974513 |
| decay | 0.0 |
| beta_1 | 0.8999999761581421 |
| beta_2 | 0.9990000128746033 |
| epsilon | 1e-07 |
| amsgrad | False |
| weight_decay | 9.999999747378752e-05 |
| exclude_from_weight_decay | None |
| training_precision | float32 |
Training Metrics
After 20 Epocs, the test accuracy of the model is 55.9% and the Top 5 test accuracy is 95.06%
Model Plot
View Model Plot
Runs of keras-io learning_to_tokenize_in_ViT on huggingface.co
4
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
-1
30-day runs
More Information About learning_to_tokenize_in_ViT huggingface.co Model
Url of learning_to_tokenize_in_ViT
learning_to_tokenize_in_ViT huggingface.co Url
Provider of learning_to_tokenize_in_ViT huggingface.co
Other API from keras-io
Updated:July 05 2024
Total runs:
108
Run Growth:
89
Growth Rate:
83.96%
Total runs:
70
Run Growth:
40
Growth Rate:
55.56%
Total runs:
54
Run Growth:
20
Growth Rate:
37.04%
Total runs:
44
Run Growth:
13
Growth Rate:
29.55%
Total runs:
41
Run Growth:
21
Growth Rate:
51.22%
Total runs:
39
Run Growth:
-39
Growth Rate:
-95.12%
Total runs:
39
Run Growth:
27
Growth Rate:
65.85%
Total runs:
39
Run Growth:
-2
Growth Rate:
-4.88%
Total runs:
38
Run Growth:
-4
Growth Rate:
-11.43%
Total runs:
37
Run Growth:
19
Growth Rate:
51.35%
Total runs:
36
Run Growth:
7
Growth Rate:
20.00%
Total runs:
34
Run Growth:
5
Growth Rate:
15.63%
Total runs:
33
Run Growth:
-12
Growth Rate:
-36.36%
Total runs:
33
Run Growth:
-23
Growth Rate:
-63.89%
Total runs:
33
Run Growth:
16
Growth Rate:
48.48%
Total runs:
31
Run Growth:
-37
Growth Rate:
-119.35%
Total runs:
29
Run Growth:
-36
Growth Rate:
-128.57%
Total runs:
29
Run Growth:
-14
Growth Rate:
-48.28%
Total runs:
29
Run Growth:
6
Growth Rate:
20.69%
Total runs:
28
Run Growth:
-15
Growth Rate:
-53.57%
Total runs:
27
Run Growth:
-11
Growth Rate:
-40.74%
Total runs:
25
Run Growth:
-7
Growth Rate:
-26.92%
Total runs:
24
Run Growth:
-8
Growth Rate:
-30.77%
Total runs:
22
Run Growth:
2
Growth Rate:
10.00%
Total runs:
22
Run Growth:
-2
Growth Rate:
-9.09%
Total runs:
20
Run Growth:
-2
Growth Rate:
-9.52%
Total runs:
19
Run Growth:
4
Growth Rate:
21.05%
Total runs:
19
Run Growth:
-9
Growth Rate:
-45.00%
Total runs:
18
Run Growth:
-1
Growth Rate:
-6.25%
Total runs:
18
Run Growth:
16
Growth Rate:
88.89%
Total runs:
17
Run Growth:
6
Growth Rate:
35.29%
Total runs:
16
Run Growth:
-7
Growth Rate:
-43.75%
Total runs:
16
Run Growth:
-8
Growth Rate:
-50.00%
Total runs:
15
Run Growth:
-11
Growth Rate:
-73.33%
Total runs:
15
Run Growth:
-7
Growth Rate:
-46.67%
Total runs:
15
Run Growth:
8
Growth Rate:
53.33%
Total runs:
14
Run Growth:
9
Growth Rate:
64.29%
Total runs:
14
Run Growth:
3
Growth Rate:
20.00%
Total runs:
12
Run Growth:
-6
Growth Rate:
-60.00%
Total runs:
12
Run Growth:
-5
Growth Rate:
-41.67%
Total runs:
12
Run Growth:
-16
Growth Rate:
-133.33%
Total runs:
11
Run Growth:
-16
Growth Rate:
-145.45%
Total runs:
11
Run Growth:
-15
Growth Rate:
-150.00%
Total runs:
10
Run Growth:
5
Growth Rate:
50.00%
Total runs:
10
Run Growth:
-7
Growth Rate:
-70.00%
Total runs:
10
Run Growth:
-16
Growth Rate:
-160.00%
Total runs:
9
Run Growth:
-3
Growth Rate:
-33.33%
Total runs:
8
Run Growth:
-5
Growth Rate:
-71.43%
Total runs:
7
Run Growth:
3
Growth Rate:
42.86%
Total runs:
6
Run Growth:
-6
Growth Rate:
-100.00%
Total runs:
5
Run Growth:
-5
Growth Rate:
-100.00%
Total runs:
4
Run Growth:
3
Growth Rate:
75.00%
Total runs:
3
Run Growth:
-16
Growth Rate:
-533.33%
Total runs:
3
Run Growth:
1
Growth Rate:
33.33%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%