community pool / model answer

gemma-4-12b-it-GGUF

chat · unsloth/gemma-4-12b-it-GGUF

50 tok/s · model median · 10 runs in the model record.

Reference rig	Quantization	Median result	Basis	TTFT (ms)	Peak VRAM	Max context	Engines
Radeon AI Pro R9700 32GB	4-bit	58.2 tok/s	reported	218	- GB	2,048	llama.cpp
RTX 2080 Ti 11GB ×2	8-bit	20.8 tok/s	reported	1,394	20 GB	2,048	llama.cpp
RTX 3060 12GB	4-bit	37 tok/s	reported	-	7.6 GB	32,768	llama.cpp
RTX 3090 24GB ×2	8-bit	48.7 tok/s	measured	349	- GB	131,072	llama.cpp
RTX 5060 Ti 16GB	4-bit	51.3 tok/s	reported	227	- GB	2,048	llama.cpp
RTX 5070 12GB	5-bit	62.8 tok/s	reported	168	- GB	2,048	llama.cpp
RTX 5070 12GB	6-bit	51.3 tok/s	reported	185	- GB	2,048	llama.cpp
Ryzen AI Max 385 24GB	4-bit	8.6 tok/s	reported	-	- GB	2,048	llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.