Pingala-8 by Genrius AI
Pingala-8 guides

Qwen3.5-2B under 1 GB: 2-bit GGUFs measured

Genrius AI guide. . About 5 minutes.

Measured: six 2-bit Qwen3.5-2B files, all under 1 GB

Pingala-8 v2 puts Qwen3.5-2B in 0.78 GB at WikiText-2 perplexity 14.06, better than unsloth UD-IQ2_M (0.86 GB, 14.96). DASH-Q is better than ours: IQ2_M 13.33, IQ2_XS 13.90.

2-bit fileSizePPLAgainst Pingala-8 v2
DASH-Q IQ2_M0.79 GB13.33Best of the six. DASH-Q better: v2 wins 3/70, +0.053
DASH-Q IQ2_XS0.76 GB13.90DASH-Q better: v2 wins 21/70, +0.012 +/- 0.007
Pingala-8 v2 (ours)0.78 GB14.06
unsloth UD-IQ2_M0.86 GB14.96v2 better on 67/70, -0.062 +/- 0.009
Pingala-8 v1 (ours)0.78 GB15.62v2 better on 70/70, -0.104
unsloth UD-IQ2_XXS0.77 GB23.41v2 better on 70/70

WikiText-2 test perplexity (PPL), 72 windows of 4096 tokens, every file on the same windows; lower is better. Last column: Pingala-8 version 2 minus that file, mean nats per token over 70 paired windows (positive means the other file is better), and the number of those windows on which version 2 has the lower loss. Exact sizes: DASH-Q IQ2_M 786,259,840 bytes, DASH-Q IQ2_XS 757,096,320, Pingala-8 v2 778,189,408, unsloth UD-IQ2_M 859,857,152, Pingala-8 v1 778,744,192, unsloth UD-IQ2_XXS 768,270,592. GB = 109 bytes. Per-window data: eval dataset; ranking: 2-bit GGUF leaderboard.

  • Model: Genrius-AI/Qwen3.5-2B-Pingala8-2bit on Hugging Face, file Qwen3.5-2B-Pingala8-2bit-v2.gguf. Text model only.
  • Runtime: version 2 needs the patched Pingala-8 llama.cpp runtime (CUDA). Stock llama.cpp, Ollama and LM Studio cannot load it. The one-click launcher still downloads version 1; for version 2, download the v2 file and pass it with -m (How to run).

In short

  • All six 2-bit files are between 0.76 and 0.86 GB. That is the file size; the KV cache and working buffers come on top, and this page does not measure them.
  • DASH-Q IQ2_M has the best perplexity of the six (13.33). DASH-Q IQ2_XS is second (13.90) and is also the smallest file.
  • Pingala-8 version 2 is third (14.06). It is better than both unsloth files and than our own version 1, at the same size as version 1.
  • The DASH-Q and unsloth files run in stock llama.cpp. Ours needs our patched CUDA runtime.

How it was measured

  • Test: llama.cpp perplexity on the WikiText-2 test set, 72 windows of 4096 tokens. The paired comparison in the table uses 70 windows: for each window we take the loss of version 2 minus the loss of the other file, then average.
  • Files: the published DASH-Q (jkim96) and unsloth GGUF files of Qwen3.5-2B, unchanged, and our two Pingala-8 files. Sizes are the exact file sizes in bytes.
  • Rival runtime: the DASH-Q and unsloth files ran on llama.cpp upstream at commit 889f2ec. On our machine (RTX 5070 Ti Laptop GPU) the IQ1_S, IQ2_S and IQ3_S CUDA kernels failed test-backend-ops, so those tensors ran on the CPU; the placement for each file is listed in the eval dataset. What we saw and the workaround: NaN perplexity with IQ1_S, IQ2_S or IQ3_S GGUFs on an RTX 50 laptop GPU.
  • Our runtime: both Pingala-8 files ran in the patched Pingala-8 llama.cpp runtime (CUDA).
  • Not measured here: speed, VRAM in use, and quiz accuracy for version 2.
  • Raw data: per-window losses for every file are in the 2-bit GGUF WikiText-2 eval dataset (summary.csv and windows.csv), and the ranking is on the 2-bit GGUF leaderboard. Its last columns compare each file with the top file over 72 windows; the table above compares with Pingala-8 version 2 over 70 paired windows.

Which file should you pick?

  • Lowest perplexity under 1 GB: DASH-Q IQ2_M (0.79 GB, 13.33). If you use an RTX 50 GPU, check its IQ2_S and IQ3_S kernels first (how).
  • Smallest file: DASH-Q IQ2_XS (0.76 GB, 13.90). It is also better than ours.
  • You already run the Pingala-8 runtime: use version 2 (0.78 GB, 14.06) instead of version 1 (15.62). Same size, better on every window.
  • Stock llama.cpp, Ollama or LM Studio: a DASH-Q or unsloth file. Ours will not load there.

Pingala-8 and this model

Pingala-8 is a 2-bit format for llama.cpp at IQ2_XXS size: an 8-state trellis code over groups of four weights, after a Hadamard rotation. Version 2 keeps the coder of version 1 and changes how bits are spread across tensors, which took Qwen3.5-2B from 15.62 to 14.06 at the same size. The method is in the paper, DOI 10.5281/zenodo.23481757; the version 2 numbers on this page are new and not in the paper.

The same version 2 recipe on a larger model: Qwen3.8-27B in 8.31 GB runs fully on one 12 GB GPU at 37.7 tokens/s, with WikiText-2 perplexity 6.89. There too DASH-Q IQ2_XS is better (6.54), and unsloth UD-IQ2_S scores 7.34.

Questions people ask

Which 2-bit GGUF of Qwen3.5-2B under 1 GB has the lowest perplexity?

Of the six files we measured, DASH-Q IQ2_M: WikiText-2 perplexity 13.33 in 0.79 GB (786,259,840 bytes). DASH-Q IQ2_XS is second at 13.90 in 0.76 GB. Our Pingala-8 version 2 is third at 14.06 in 0.78 GB.

Is Pingala-8 version 2 better than DASH-Q on Qwen3.5-2B?

No. DASH-Q IQ2_XS (0.76 GB, 13.90) is better: version 2 has the lower loss on only 21 of 70 windows, +0.012 +/- 0.007 nats per token, and the DASH-Q file is 21 MB smaller. DASH-Q IQ2_M (0.79 GB, 13.33) is better too: version 2 has the lower loss on only 3 of 70 windows, +0.053 nats per token.

Is Pingala-8 version 2 better than unsloth's 2-bit Qwen3.5-2B files?

On WikiText-2, yes. Against unsloth UD-IQ2_M (0.86 GB, 14.96), version 2 (0.78 GB, 14.06) is better on 67 of 70 windows, -0.062 +/- 0.009 nats per token, in a file 82 MB smaller. Against unsloth UD-IQ2_XXS (0.77 GB, 23.41) it is better on 70 of 70 windows.

Does a Qwen3.5-2B file under 1 GB fit a 1 GB GPU?

Not necessarily. Under 1 GB here means the file size: all six files are 0.76 to 0.86 GB. Running a model also needs memory for the KV cache and the runtime's working buffers, which this page does not measure. Our GPU fit calculator gives an estimate.

Can I run the version 2 file in Ollama, LM Studio or stock llama.cpp?

No. It needs the patched Pingala-8 llama.cpp runtime (CUDA, NVIDIA RTX 30, 40 or 50 series). The one-click launcher still downloads the version 1 file; to use version 2, download Qwen3.5-2B-Pingala8-2bit-v2.gguf from Hugging Face and pass it with -m. The DASH-Q and unsloth files run in stock llama.cpp.

What changed from version 1 to version 2?

How the bits are spread across tensors. The size stays the same (778,189,408 bytes against 778,744,192) and WikiText-2 perplexity goes from 15.62 to 14.06: version 2 is better on 70 of 70 windows, -0.104 nats per token.

More guides