community pool / model answer

nvidia_Nemotron-Cascade-2-30B-A3B-GGUF

chat · bartowski/nvidia_Nemotron-Cascade-2-30B-A3B-GGUF

164.9 tok/s · model median · 16 runs in the model record.

Reference rig	Quantization	Median result	Basis	TTFT (ms)	Peak VRAM	Max context	Engines
Arc Pro B70 32GB	4-bit	50.9 tok/s	reported	449	- GB	2,048	llama.cpp
M1 Max 64GB	4-bit	57.5 tok/s	reported	50	- GB	2,048	ollama
Minisforum UM790 Pro 64GB	4-bit	24.8 tok/s	reported	-	- GB	2,048	llama.cpp
Radeon 890M 32GB	4-bit	21.5 tok/s	reported	1,430	- GB	2,048	llama.cpp
RTX 3090 24GB	4-bit	190.3 tok/s	measured	1,769	- GB	131,072	llama.cpp
Ryzen AI Max 385 24GB	4-bit	21.4 tok/s	reported	1,430	- GB	2,048	llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.