keras / segformer_b5_ade20k_640

Model Overview

The SegFormer model was proposed in SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers by Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, Ping Luo. The model consists of a hierarchical Transformer encoder and a lightweight all-MLP decode head to achieve great results on image segmentation benchmarks such as ADE20K and Cityscapes.

` Weights are released under the MIT License . Keras model code is released under the Apache 2 License .

Links

• Segmentation Quickstart Notebook
• SegFormer API Documentation
• SegFormer Model Card

Installation

Keras and KerasCV can be installed with:

pip install -U -q keras-cv
pip install -U -q keras>=3

Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the Keras Getting Started page.

Presets

The following model checkpoints are provided by the Keras team. Full code examples for each are available below.

Example code

import keras_cv

images = np.ones(shape=(1, 224, 224, 3))
labels = np.zeros(shape=(1, 224, 224, 1))


model = keras_cv.models.SegFormer.from_preset(
            "segformer_b0", num_classes=2
        )
# Evaluate model
model(images)

Example Usage

import keras_cv
import  keras
import numpy as np

Using the class with a backbone :

import tensorflow as tf
import keras_cv

images = np.ones(shape=(1, 96, 96, 3))
labels = np.zeros(shape=(1, 96, 96, 1))
backbone = keras_cv.models.MiTBackbone.from_preset("segformer_b5_ade20k_640")
model = keras_cv.models.segmentation.SegFormer(
    num_classes=1, backbone=backbone,
)

# Evaluate model
model(images)

# Train model
model.compile(
    optimizer="adam",
    loss=keras.losses.BinaryCrossentropy(from_logits=False),
    metrics=["accuracy"],
)
model.fit(images, labels, epochs=3)

Example Usage with Hugging Face URI

import keras_cv
import  keras
import numpy as np

Using the class with a backbone :

import tensorflow as tf
import keras_cv

images = np.ones(shape=(1, 96, 96, 3))
labels = np.zeros(shape=(1, 96, 96, 1))
backbone = keras_cv.models.MiTBackbone.from_preset("hf://keras/segformer_b5_ade20k_640")
model = keras_cv.models.segmentation.SegFormer(
    num_classes=1, backbone=backbone,
)

# Evaluate model
model(images)

# Train model
model.compile(
    optimizer="adam",
    loss=keras.losses.BinaryCrossentropy(from_logits=False),
    metrics=["accuracy"],
)
model.fit(images, labels, epochs=3)