Neural Super Sampling (NSS) is an innovative, efficient network for temporal super sampling on mobile devices. Content rendered at 540p can be upscaled to 1080p, resulting in up to 50% GPU savings. With our retraining tools content creators and game studios can build derivatives of the model suited to artwork style and performance requirements.
🎥 Neural Super Sampling Demo
Model Details
Neural Super Sampling (NSS) is a parameter prediction model for real-time temporal super sampling developed by Arm, optimized for execution on Neural Accelerators (NX) in mobile GPUs. It enables high-resolution rendering at a lower compute cost by reconstructing high-quality output frames from low-resolution temporal inputs. NSS is particularly suited for mobile gaming, XR, and other power-constrained graphics use cases.
NSS is under active development with regular updates planned. As we increase the size and diversity of the training dataset we expect to see significant quality improvements. Follow Arm to stay up to date on the latest releases.
The model is released under Arm's
AI Model Community License
which allows NSS to be retrained on datasets captured from your own content. Future releases of the
Neural Graphics Model Gym
will provide the tools to capture and convert content for use in (re)retraining..
Uses
NSS can be directly integrated into graphics pipelines using ML extensions for Vulkan®. See included ML SDK for Vulkan
scenario
for the simplest way to evaluate the model. The scenario includes the necessary pre- and post-processing compute shaders along with a single frame worth of input data.
The recommended way of integrating the model into a graphics pipeline is by using the
VGF Library
from the ML SDK for Vulkan.
NSS is released under a
permissive license
designed to foster innovation in the graphics industry and provide differentiation to content creators.
Not suited for non-temporal tasks such as a standalone image upsampling
Bias, Risks, and Limitations
Requires accurate motion vectors and frame history for stable output
May underperform in extremely low framerate scenarios (<10 FPS) with fast camera movement
Padding of the input is needed if input dimensions are not divisible by 8
Recommendations
For ultra-low-FPS use cases, reduce the camera speed, acceleration, or both so that the relative motion between frames mimics the
application running at a higher frame rate.
How to Get Started with the Model
This repository contains pre-trained weights and compiled NSS model in VGF format ready for integration with Vulkan applications.
The included Scenario demonstrates full execution of the model on a Vulkan compute-capable system. An Emulation Layer is provided to implement ML Extensions for Vulkan where it is not supported by the native Vulkan driver.
bin\windows-x86_64\scenario-runner.exe --scenario scenario\scenario.json --output out
On Linux:
bin/linux-x86_64/scenario-runner--scenario scenario/scenario.json --output out
Output images are encoded as
B10G11R11_UFLOAT
. This format is common for framebuffers but not widely supported by image viewers. Use
RenderDoc
to view these images.
neural-super-sampling huggingface.co is an AI model on huggingface.co that provides neural-super-sampling's model effect (), which can be used instantly with this Arm neural-super-sampling model. huggingface.co supports a free trial of the neural-super-sampling model, and also provides paid use of the neural-super-sampling. Support call neural-super-sampling model through api, including Node.js, Python, http.
neural-super-sampling huggingface.co is an online trial and call api platform, which integrates neural-super-sampling's modeling effects, including api services, and provides a free online trial of neural-super-sampling, you can try neural-super-sampling online for free by clicking the link below.
Arm neural-super-sampling online free url in huggingface.co:
neural-super-sampling is an open source model from GitHub that offers a free installation service, and any user can find neural-super-sampling on GitHub to install. At the same time, huggingface.co provides the effect of neural-super-sampling install, users can directly use neural-super-sampling installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
neural-super-sampling install url in huggingface.co: