Intel / distilbert-base-uncased-distilled-squad-int8-static-inc

huggingface.co
Total runs: 311
24-hour runs: 0
7-day runs: -58
30-day runs: -791
Model's Last Updated: March 29 2024
question-answering

Introduction of distilbert-base-uncased-distilled-squad-int8-static-inc

Model Details of distilbert-base-uncased-distilled-squad-int8-static-inc

Model Card for INT8 DistilBERT Base Uncased Fine-Tuned on SQuAD

This model is an INT8 quantized version of DistilBERT base uncased, which has been fine-tuned on the Stanford Question Answering Dataset (SQuAD). The quantization was performed using the Hugging Face's Optimum-Intel, leveraging the Intel® Neural Compressor.

Model Detail Description
Model Authors Xin He Zixuan Cheng Yu Wenz
Date Aug 4, 2022
Version The base model for this quantization process was distilbert-base-uncased-distilled-squad, a distilled version of BERT designed for the question-answering task.
Type Language Model
Paper or Other Resources Base Model: distilbert-base-uncased-distilled-squad
License apache-2.0
Questions or Comments Community Tab and Intel DevHub Discord
Quantization Details The model underwent post-training static quantization to convert it from its original FP32 precision to INT8, optimizing for size and inference speed while aiming to retain as much of the original model's accuracy as possible.
Calibration Details For PyTorch, the calibration dataloader was the train dataloader with a real sampling size of 304 due to the default calibration sampling size of 300 not being exactly divisible by the batch size of 8. For the ONNX version, the calibration was performed using the eval dataloader with a default calibration sampling size of 100.
Intended Use Description
Primary intended uses This model is intended for question-answering tasks, where it can provide answers to questions given a context passage. It is optimized for scenarios requiring fast inference and reduced model size without significantly compromising accuracy.
Primary intended users Researchers, developers, and enterprises that require efficient, low-latency question answering capabilities in their applications, particularly where computational resources are limited.
Out-of-scope uses

Evaluation

PyTorch Version

This is an INT8 PyTorch model quantized with huggingface/optimum-intel through the usage of Intel® Neural Compressor .

INT8 FP32
Accuracy (eval-f1) 86.1069 86.8374
Model size (MB) 74.7 265
ONNX Version

This is an INT8 ONNX model quantized with Intel® Neural Compressor .

INT8 FP32
Accuracy (eval-f1) 0.8633 0.8687
Model size (MB) 154 254

Usage

Optimum Intel w/ Neural Compressor

from optimum.intel import INCModelForQuestionAnswering

model_id = "Intel/distilbert-base-uncased-distilled-squad-int8-static"
int8_model = INCModelForQuestionAnswering.from_pretrained(model_id)

Optimum w/ ONNX Runtime

from optimum.onnxruntime import ORTModelForQuestionAnswering
model = ORTModelForQuestionAnswering.from_pretrained('Intel/distilbert-base-uncased-distilled-squad-int8-static')

Ethical Considerations

While not explicitly mentioned, users should be aware of potential biases present in the training data (SQuAD and Wikipedia), and consider the implications of these biases on the model's outputs. Additionally, quantization may introduce or exacerbate biases in certain scenarios.

Caveats and Recommendations

  • Users should consider the balance between performance and accuracy when deploying quantized models in critical applications.
  • Further fine-tuning or calibration may be necessary for specific use cases or to meet stricter accuracy requirements.

Runs of Intel distilbert-base-uncased-distilled-squad-int8-static-inc on huggingface.co

311
Total runs
0
24-hour runs
-22
3-day runs
-58
7-day runs
-791
30-day runs

More Information About distilbert-base-uncased-distilled-squad-int8-static-inc huggingface.co Model

More distilbert-base-uncased-distilled-squad-int8-static-inc license Visit here:

https://choosealicense.com/licenses/apache-2.0

distilbert-base-uncased-distilled-squad-int8-static-inc huggingface.co

distilbert-base-uncased-distilled-squad-int8-static-inc huggingface.co is an AI model on huggingface.co that provides distilbert-base-uncased-distilled-squad-int8-static-inc's model effect (), which can be used instantly with this Intel distilbert-base-uncased-distilled-squad-int8-static-inc model. huggingface.co supports a free trial of the distilbert-base-uncased-distilled-squad-int8-static-inc model, and also provides paid use of the distilbert-base-uncased-distilled-squad-int8-static-inc. Support call distilbert-base-uncased-distilled-squad-int8-static-inc model through api, including Node.js, Python, http.

distilbert-base-uncased-distilled-squad-int8-static-inc huggingface.co Url

https://huggingface.co/Intel/distilbert-base-uncased-distilled-squad-int8-static-inc

Intel distilbert-base-uncased-distilled-squad-int8-static-inc online free

distilbert-base-uncased-distilled-squad-int8-static-inc huggingface.co is an online trial and call api platform, which integrates distilbert-base-uncased-distilled-squad-int8-static-inc's modeling effects, including api services, and provides a free online trial of distilbert-base-uncased-distilled-squad-int8-static-inc, you can try distilbert-base-uncased-distilled-squad-int8-static-inc online for free by clicking the link below.

Intel distilbert-base-uncased-distilled-squad-int8-static-inc online free url in huggingface.co:

https://huggingface.co/Intel/distilbert-base-uncased-distilled-squad-int8-static-inc

distilbert-base-uncased-distilled-squad-int8-static-inc install

distilbert-base-uncased-distilled-squad-int8-static-inc is an open source model from GitHub that offers a free installation service, and any user can find distilbert-base-uncased-distilled-squad-int8-static-inc on GitHub to install. At the same time, huggingface.co provides the effect of distilbert-base-uncased-distilled-squad-int8-static-inc install, users can directly use distilbert-base-uncased-distilled-squad-int8-static-inc installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

distilbert-base-uncased-distilled-squad-int8-static-inc install url in huggingface.co:

https://huggingface.co/Intel/distilbert-base-uncased-distilled-squad-int8-static-inc

Url of distilbert-base-uncased-distilled-squad-int8-static-inc

distilbert-base-uncased-distilled-squad-int8-static-inc huggingface.co Url

Provider of distilbert-base-uncased-distilled-squad-int8-static-inc huggingface.co

Intel
ORGANIZATIONS

Other API from Intel

huggingface.co

Total runs: 781.0K
Run Growth: 348.1K
Growth Rate: 44.58%
Updated:February 09 2024
huggingface.co

Total runs: 41.1K
Run Growth: -7.2K
Growth Rate: -17.45%
Updated:February 24 2024
huggingface.co

Total runs: 27.4K
Run Growth: 2.8K
Growth Rate: 10.39%
Updated:March 29 2024
huggingface.co

Total runs: 15.5K
Run Growth: 5.6K
Growth Rate: 35.72%
Updated:February 24 2025
huggingface.co

Total runs: 1.8K
Run Growth: -2.0K
Growth Rate: -110.29%
Updated:March 26 2024
huggingface.co

Total runs: 1.5K
Run Growth: 284
Growth Rate: 18.92%
Updated:May 20 2024
huggingface.co

Total runs: 871
Run Growth: 184
Growth Rate: 21.75%
Updated:June 12 2024