community pool / model answer

Bonsai-27B-gguf

chat · prism-ml/Bonsai-27B-gguf

31.5 tok/s · model median · 36 runs in the model record.

Reference rig	Quantization	Median result	Basis	TTFT (ms)	Peak VRAM	Max context	Engines
RTX 3060 12GB ×2	4-bit	29.3 tok/s	measured	1	21.9 GB	262,144	llama.cpp
RTX 3080 12GB	1-bit	72.9 tok/s	reported	2,054	5.1 GB	32,768	llama.cpp
RTX 3090 24GB	1-bit	39.8 tok/s	measured	31,904	5.6 GB	262,144	llama.cpp
RTX 5090 32GB	1-bit	136.7 tok/s	reported	82	- GB	2,048	lmstudio
RX 7700 XT 12GB	1-bit	44 tok/s	reported	117	- GB	2,048	vllm
Ryzen AI Max 395 128GB	1-bit	33.6 tok/s	measured	26,241	- GB	262,144	llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.