community pool / model answer

NVIDIA-Nemotron-Nano-9B-v2

chat · nvidia/NVIDIA-Nemotron-Nano-9B-v2

64.3 tok/s · model median · 22 runs in the model record.

Reference rig	Quantization	Median result	Basis	TTFT (ms)	Peak VRAM	Max context	Engines
RTX 3060 12GB ×2	4-bit	45.2 tok/s	measured	6,144	8.6 GB	131,072	llama.cpp
RTX 3060 + RTX 2080 20GB	4-bit	50 tok/s	reported	-	5.2 GB	2,048	llama.cpp
RTX 3060 Ti 8GB	4-bit	64.3 tok/s	reported	292	- GB	2,048	llama.cpp
RTX 3080 12GB	4-bit	98 tok/s	reported	-	5.2 GB	2,048	llama.cpp
RTX 3090 24GB	4-bit	73.7 tok/s	measured	3,388	- GB	131,072	llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.