community pool / model answer

gemma-4-12B-it

chat · google/gemma-4-12B-it

45.2 tok/s · model median · 38 runs in the model record.

Reference rig	Quantization	Median result	Basis	TTFT (ms)	Peak VRAM	Max context	Engines
M1 Max 64GB	16-bit	14 tok/s	reported	2,324	- GB	794	llama.cpp
Radeon 890M 32GB	4-bit	10.4 tok/s	reported	-	- GB	4,096	llama.cpp
Radeon 890M 32GB	6-bit	8.5 tok/s	reported	-	- GB	4,096	llama.cpp
Radeon 890M 32GB	8-bit	6.8 tok/s	reported	-	- GB	4,096	llama.cpp
Radeon AI Pro R9700 32GB ×3	8-bit	36.7 tok/s	reported	454	15.3 GB	794	llama.cpp
Radeon AI Pro R9700 32GB ×3	16-bit	19.3 tok/s	reported	1,448	- GB	794	llama.cpp
RTX 3060 12GB	4-bit	76.7 tok/s	measured	589	9.8 GB	262,144	llama.cpp
RTX 3090 24GB	4-bit	52.3 tok/s	measured	4,752	0 GB	131,072	llama.cpp
RTX 5000 16GB	6-bit	21.5 tok/s	reported	-	14 GB	262,144	llama.cpp
RTX 5000 16GB	8-bit	23.4 tok/s	reported	-	14.9 GB	98,304	llama.cpp
RTX 5090 32GB	8-bit	87.2 tok/s	reported	85	13.7 GB	8,192	llama.cpp
RTX PRO 6000 Blackwell 96GB	16-bit	86.9 tok/s	reported	44	- GB	2,048	vllm
RTX PRO 6000 Blackwell 96GB ×2	16-bit	132.2 tok/s	reported	29	- GB	2,048	vllm
RTX PRO 6000 Blackwell 96GB ×4	16-bit	145.7 tok/s	reported	35	- GB	2,048	vllm
RX 570 4GB ×2	4-bit	18.9 tok/s	measured	642	- GB	2,048	llama.cpp
RX 6700 XT 12GB	4-bit	37.8 tok/s	measured	-	- GB	4,736	llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.