community pool / model answer

LFM2.5-8B-A1B-GGUF

chat · LiquidAI/LFM2.5-8B-A1B-GGUF

218.2 tok/s · model median · 18 runs in the model record.

Reference rig	Quantization	Median result	Basis	TTFT (ms)	Peak VRAM	Max context	Engines
AMD Ryzen 5 7640U	4-bit	22.3 tok/s	reported	577	- GB	2,048	vllm
RTX 3060 12GB ×2	4-bit	218.2 tok/s	measured	-	6.4 GB	128,000	llama.cpp
RTX 3060 12GB ×2	6-bit	175.6 tok/s	reported	-	8.2 GB	128,000	llama.cpp
RTX 3080 12GB	4-bit	328.4 tok/s	reported	261	6.4 GB	128,000	llama.cpp
RTX 3080 12GB	6-bit	277.8 tok/s	reported	286	8.1 GB	128,000	llama.cpp
RTX 3090 24GB ×2	8-bit	276.3 tok/s	measured	317	- GB	128,000	llama.cpp
RX 570 4GB ×2	4-bit	160.7 tok/s	measured	1,630	- GB	2,048	llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.