Model Style not-a-Bench
A tool for quickly eyeballing the stylistic choices of various LLMs side by side, focused on models at 4-bit quantization under 100GB of weights, as a qualitative alternative to benchmark-driven model comparison.
What it addresses
Recent LLMs score highly on benchmarks yet disappoint on simple tasks or sanity checks, and benchmarks reveal nothing about the models' stylistic choices.
Shared over time
Each entry reflects what was shared on that date. Details may have changed since then.
Model Style not-a-Bench
A tool for quickly eyeballing the stylistic choices of various LLMs side by side, focused on models at 4-bit quantization under 100GB of weights, as a qualitative alternative to benchmark-driven model comparison.
Problem: Recent LLMs score highly on benchmarks yet disappoint on simple tasks or sanity checks, and benchmarks reveal nothing about the models' stylistic choices.
Posted by @techsystems · Original comment ↗
“So I made 'Model Style not-a-Bench' to be able to quickly eyeball various LLMs, most preferably at 4-bit quants, under 100GB weights (excluding storage offloads like Qwen Flash Next's approach).”