Building the Frontier of Multimodal Inference

Multimodal inference is not one request and one response. People generate, look, edit, and try again, often without knowing the image or video they want until they see it. Each output changes the next request, and it may become the input to another model or another person. The requirements follow from that. Quality must survive optimization, latency must support a live creative loop, and cost must stay low enough to pay for many attempts.

These three are coupled. Latency decides whether the result arrives while the idea is still yours. Cost decides how many attempts you can afford: cheap attempts widen the design space, expensive ones force an early commitment. Quality decides whether the output is useful at all, and it matters most when one person's result becomes the next person's input. Video presses hardest on all three. It takes two orders of magnitude more compute than an image, but nobody wants to wait two orders of magnitude longer.

Most production serving stacks were built for autoregressive language decoding: one token at a time, where each step is usually limited by memory bandwidth. Image and video models behave differently. At each generation step, they process a whole spatial or spatiotemporal latent in parallel, which is dense compute. The bottleneck moves from memory to compute, and the serving economics built for language models do not transfer. That is why we built a new stack instead of tuning an old one.

Nunchux is a lab building multimodal inference from the ground up to optimize quality, cost, and latency together. Speed does not come from a single trick. It comes from designing the Nunchux Model Optimizer and the Nunchux Engine as one system, so each gain at one layer makes room for the next. We apply that approach across NVIDIA GPUs, AMD GPUs, and mobile devices.

Quality is the corner we refuse to trade, so we measure it. Every model is checked against its original output before it enters the catalog, and one that loses fidelity stays out. Our SVDQuant runs FLUX.1-schnell 3× faster, while ImageReward rises from 0.938 to 0.966. The images come back faster with even better quality.

Nunchux grew from a collaboration between MIT HAN Lab and CMU's Generative Intelligence Lab, whose work over the past decade laid much of the foundation for efficient generative models, including SVDQuant, Sparse VideoGen, DeltaQuant, and Radial Attention, and has been cited more than 200,000 times. SVDQuant has been downloaded 4 million times and integrated into NVIDIA TensorRT, AMD Quark, vLLM, SGLang, Diffusers, and ComfyUI as the standard quantization algorithm.

The team brings together PhD students and faculty from both labs along with industry veterans who shipped production systems at NVIDIA, AMD, and Snowflake. We work out of San Francisco.

Nunchux is backed by lead investors Emergence Capital and E14 Fund, with participation from Archerman Capital, Scale Asia Ventures, Tectonic Ventures, KungHo Fund, and Plaid Matrix Fund, and founders and leadership at Black Forest Labs, Reve, HeyGen, Pika Labs, Together AI, Skypilot, and Skild AI, and founding members of Meta Superintelligence Labs.

Nunchux Modelverse opens today

The Modelverse features Nunchux-optimized models with two performance tiers: Radical Speed for the lowest latency and Radical Value for the lowest cost. The Modelverse also features Partner APIs from foundation labs: Google Veo 3.1, HeyGen Avatar 5, Kling V3, and more.

Try the models in Nunchux Modelverse. Demand is ahead of the capacity we can bring online, so we onboard developers and teams from the waitlist, and new accounts start with $1 in free credits.

If you run visual models at scale, contact sales. We are hiring. Please visit our careers page for more details.