NVIDIA RTX AI Toolkit

The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PCs and cloud. Via the end-to-end workflow , developers can customize open source models, reduce size by up to 3x, improve performance by up to 4x, and seamlessly deploy within their applications to 100M RTX AI PCs.

Get Started

A workflow diagram of the NVIDIA AI workbench

The Benefits of NVIDIA RTX AI Toolkit

A decorative image representing an integrated, GPU-accelerated workflow

Accelerated End-to-End Workflow

Go from model customization to optimization to Windows PC deployment in an integrated, GPU-accelerated workflow managed via NVIDIA AI Workbench.

A decorative image representing an integration with the developer ecosystem

Integrated With the Developer Ecosystem

Models that are customized and optimized with the RTX AI Toolkit can be run in popular ecosystem tools like Jan.AI and OobaBooga, and are compatible with frameworks like LangChain and LlamaIndex.

Optimized For Top Performance

Choose from a variety of inference backends including TensorRT10.0, TensorRT-LLM, DirectML, Llama.cpp, and more for top inference performance on RTX and compatibility across GPU, NPU, and CPU hardware.

A decorative image representing AI models been deployed across PC and cloud

Deploy Anywhere

Seamlessly deploy optimized AI models across PC and cloud with a unified application API using NVIDIA AIM SDK.

Explore Key Features

Orchestrate Model Tuning

Use AI Workbench to streamline, organize and execute AI model customization using purpose-built tools and SDKs with HuggingFace Transformers. AI Workbench provides a user-friendly interface on your choice of infrastructure—be it in the data center, cloud, RTX workstation or PC.

NVIDIA GPUs are been optimized to run efficiently on RTX PCs

Optimize and Build Efficient Models to Run on RTX PCs

Reduce the memory footprint of AI models by up to 3x to fit on a PC with state-of-the-art quantization techniques using NVIDIA TensorRT Model Optimizer. Then, quickly build optimized engines for any target NVIDIA GPU using TensorRT or TensorRT Cloud. These combined techniques deliver up to 4x faster inference performance. You can manage the workflow either directly or via AI Workbench.

Accelerate and Deploy Applications on RTX PCs

Integrate AI models into applications with the NVIDIA AI Inference Manager (AIM) to orchestrate deployment across local RTX PC and cloud inferencing. NVIDIA AIM simplifies the complexity of AI integration by pre-configuring the PC with the necessary AI models, engines, and dependencies, and supports all major inference backends. Developers have full control over the policy to orchestrate cloud API endpoints through NVIDIA NIMs in the cloud as well as local execution with their choice of backend - be it TensorRT / TensorRT-LLM, ONNX Runtime and DirectML, Llama.cpp, etc - through both in-process and out-of-process execution modes with C++ plugins. AIM is currently available in early access to developers, apply here now.

A software developer deploys applications on a RTX PC

Get Started With RTX AI Toolkit

RTX AI Toolkit in Action

Watch here for an end-to-end developer walkthrough of the NVIDIA RTX AI Toolkit, from model development to application deployment. This workflow showcases the model development workflow with AI Workbench and LlamaFactory—from customizing a Llama 3-7B model with the QLoRA technique to quantizing the model checkpoint with TensorRT Model Optimizer. The application deployment phase utilizes the NVIDIA AI Inference Manager (AIM) SDK.

Watch Now

NVIDIA AI Workbench

NVIDIA AI Workbench enables developers and data scientists to create, collaborate, and reproduce AI projects on infrastructure of your choice - from RTX laptops and workstations to data center and cloud. AI Workbench delivers easy GPU workstation setup for experimentation, testing, and prototyping of AI workloads across heterogeneous platforms.

Learn More About NVIDIA AI Workbench

Applications Accelerated by NVIDIA AI

DaVinci Resolve

DaVinci Resolve accelerates real-time color correction, denoising, re-timing, and image enhancement in professional editing software with the NVIDIA RTX AI Toolkit.

Learn More About DaVinci Resolve

Topaz Labs

With the NVIDIA RTX AI Toolkit, Topaz has achieved superior performance in a variety of AI tasks on RTX systems, such as photo denoising, sharpening, photo super-resolution, video slow motion, video super-resolution, video stabilization, and more for their photo and video AI applications.

Learn More About Topaz Labs