Total runs:
1.4M
Run Growth:
-235.5K
Growth Rate:
-17.15%
Sponsored by
SocQ
apple / coreml-detr-semantic-segmentation
Model's Last Updated: August 09 2024
image-segmentation
Introduction of coreml-detr-semantic-segmentation
Model Details of coreml-detr-semantic-segmentation
DETR-Resnet50 (semantic segmentation) Core ML Models
See the Files tab for converted models.
DEtection TRansformer (DETR) model trained end-to-end on COCO 2017 object detection (118k annotated images). It was introduced in the paper End-to-End Object Detection with Transformers by Carion et al. and first released in this repository .
Model description
The DETR model is an encoder-decoder transformer with a convolutional backbone. Two heads are added on top of the decoder outputs in order to perform object detection: a linear layer for the class labels and a MLP (multi-layer perceptron) for the bounding boxes. The model uses so-called object queries to detect objects in an image. Each object query looks for a particular object in the image. For COCO, the number of object queries is set to 100.
The model is trained using a "bipartite matching loss": one compares the predicted classes + bounding boxes of each of the N = 100 object queries to the ground truth annotations, padded up to the same length N (so if an image only contains 4 objects, 96 annotations will just have a "no object" as class and "no bounding box" as bounding box). The Hungarian matching algorithm is used to create an optimal one-to-one mapping between each of the N queries and each of the N annotations. Next, standard cross-entropy (for the classes) and a linear combination of the L1 and generalized IoU loss (for the bounding boxes) are used to optimize the parameters of the model.
Evaluation - Variants
| Variant | Parameters | Size (MB) | Weight Precision | Act. Precision | IoU | Pixel accuracy |
|---|---|---|---|---|---|---|
| facebook/detr-resnet-50-panoptic (PyTorch) | 43M | 172 | Float32 | Float32 | 0.393 | 0.746 |
| DETRResnet50SemanticSegmentationF32 | 43M | 171 | Float32 | Float32 | 0.393 | 0.746 |
| DETRResnet50SemanticSegmentationF16 | 43M | 86 | Float16 | Float16 | 0.395 | 0.746 |
IoU and Pixel accuracy measured on 512 images from the COCO dataset. The ground truth labels were extracted from the panoptic segmentation annotations, transformed to semantic segmentation masks. Input images were resized so that the smaller edge equals 448, then center-cropped.
Inference time
The following results refer to DETRResnet50SemanticSegmentationF16. The compute units for MacBook Pro (M1 Max) were manually selected to "CPU and Neural Engine".
| Device | OS | Inference time (ms) | Dominant compute unit |
|---|---|---|---|
| iPhone 15 Pro Max | 17.5 | 40 | Neural Engine |
| MacBook Pro (M1 Max) | 14.5 | 43 | Neural Engine |
| iPhone 12 Pro Max | 18.0 | 52 | Neural Engine |
| MacBook Pro (M3 Max) | 15.0 | 29 | Neural Engine |
Download
Install
huggingface-cli
brew install huggingface-cli
To download one of the
.mlpackage
folders to the
models
directory:
huggingface-cli download \
--local-dir models --local-dir-use-symlinks False \
apple/coreml-detr-semantic-segmentation \
--include "DETRResnet50SemanticSegmentationF16.mlpackage/*"
To download everything, skip the
--include
argument. This will retrieve
float32
and
float16
variants, as well as quantized versions of the
float16
variant.
Integrate in Swift apps
The
huggingface/coreml-examples
repository contains sample Swift code for
coreml-detr-semantic-segmentation
and other models. See
the instructions there
to build the demo app, which shows how to use the model in your own Swift apps.
Runs of apple coreml-detr-semantic-segmentation on huggingface.co
151
Total runs
-2
24-hour runs
2
3-day runs
8
7-day runs
57
30-day runs
More Information About coreml-detr-semantic-segmentation huggingface.co Model
Url of coreml-detr-semantic-segmentation
coreml-detr-semantic-segmentation huggingface.co Url
Provider of coreml-detr-semantic-segmentation huggingface.co
Other API from apple
Updated:März 01 2025
Total runs:
500.5K
Run Growth:
-1.0M
Growth Rate:
-195.49%
Total runs:
336.6K
Run Growth:
-422.8K
Growth Rate:
-123.34%
Total runs:
41.2K
Run Growth:
-166.5K
Growth Rate:
-339.30%
Total runs:
25.1K
Run Growth:
22.6K
Growth Rate:
93.68%
Total runs:
20.3K
Run Growth:
15.4K
Growth Rate:
76.01%
Total runs:
9.9K
Run Growth:
-64.9K
Growth Rate:
-660.52%
Total runs:
8.3K
Run Growth:
-4.8K
Growth Rate:
-56.91%
Total runs:
7.6K
Run Growth:
-939.2K
Growth Rate:
-12115.39%
Total runs:
4.6K
Run Growth:
1.6K
Growth Rate:
35.32%
Total runs:
4.5K
Run Growth:
-339
Growth Rate:
-7.80%
Total runs:
2.6K
Run Growth:
1.7K
Growth Rate:
60.13%
Total runs:
2.5K
Run Growth:
2.1K
Growth Rate:
81.65%
Total runs:
1.8K
Run Growth:
478
Growth Rate:
25.51%
Total runs:
1.5K
Run Growth:
13
Growth Rate:
0.85%
Total runs:
1.4K
Run Growth:
-104
Growth Rate:
-7.54%
Total runs:
1.2K
Run Growth:
-159
Growth Rate:
-13.54%
Total runs:
1.1K
Run Growth:
372
Growth Rate:
31.42%
Total runs:
1.1K
Run Growth:
189
Growth Rate:
17.86%
Total runs:
921
Run Growth:
-24.8K
Growth Rate:
-2823.23%
Total runs:
909
Run Growth:
26
Growth Rate:
2.78%
Total runs:
809
Run Growth:
-28
Growth Rate:
-3.44%
Total runs:
617
Run Growth:
-254
Growth Rate:
-42.33%
Total runs:
510
Run Growth:
239
Growth Rate:
46.95%
Total runs:
463
Run Growth:
138
Growth Rate:
29.61%
Total runs:
438
Run Growth:
234
Growth Rate:
53.42%
Total runs:
434
Run Growth:
-625
Growth Rate:
-147.75%
Total runs:
403
Run Growth:
-117
Growth Rate:
-29.25%
Total runs:
381
Run Growth:
146
Growth Rate:
37.73%
Total runs:
256
Run Growth:
44
Growth Rate:
17.25%
Total runs:
226
Run Growth:
80
Growth Rate:
36.87%
Total runs:
216
Run Growth:
80
Growth Rate:
40.61%
Total runs:
188
Run Growth:
59
Growth Rate:
31.38%
Total runs:
177
Run Growth:
48
Growth Rate:
27.59%
Total runs:
165
Run Growth:
103
Growth Rate:
66.03%
Total runs:
148
Run Growth:
-42
Growth Rate:
-28.38%
Total runs:
145
Run Growth:
-64
Growth Rate:
-45.39%
Total runs:
138
Run Growth:
66
Growth Rate:
47.83%
Total runs:
133
Run Growth:
109
Growth Rate:
81.34%
Total runs:
130
Run Growth:
-28
Growth Rate:
-21.71%
Total runs:
124
Run Growth:
27
Growth Rate:
21.09%
Total runs:
102
Run Growth:
-114
Growth Rate:
-114.00%
Total runs:
102
Run Growth:
61
Growth Rate:
59.80%
Total runs:
97
Run Growth:
21
Growth Rate:
21.21%
Total runs:
87
Run Growth:
-10
Growth Rate:
-10.87%
Total runs:
81
Run Growth:
34
Growth Rate:
43.04%
Total runs:
60
Run Growth:
-514
Growth Rate:
-842.62%
Total runs:
58
Run Growth:
24
Growth Rate:
42.11%
Total runs:
57
Run Growth:
26
Growth Rate:
68.42%
Total runs:
49
Run Growth:
15
Growth Rate:
29.41%
Total runs:
36
Run Growth:
-41
Growth Rate:
-110.81%
Total runs:
35
Run Growth:
11
Growth Rate:
31.43%
Total runs:
29
Run Growth:
-19
Growth Rate:
-70.37%