Pingala-8 by Genrius AINew and updated pages on the Pingala-8 site: 2-bit compression for local LLMs in llama.cpp, the paper, and guides to running LLMs on small GPUs.tag:genrius-ai.github.io,2026-10-24:pingala8-llm/feed2026-10-24T17:53:59ZGenrius AIhttps://github.com/Genrius-AIPingala-8 is open source under the Apache License 2.0.Pingala-8: 85% smaller 2-bit LLMs for llama.cpptag:genrius-ai.github.io,2026-10-24:pingala8-llm/index2026-10-24T13:12:23Z2026-10-24T17:51:46ZPingala-8 compresses Qwen3-4B from 8.05 GB to 1.24 GB. At the size of llama.cpp's 2-bit IQ2_XXS it scores 21% higher on five quizzes. Open code, one-click Windows build.How we fit Qwen3-4B into 1.24 GB without retrainingtag:genrius-ai.github.io,2026-10-24:pingala8-llm/article.html2026-10-24T13:12:23Z2026-10-24T17:28:54ZGenrius AI's Pingala-8 is a 2-bit format for llama.cpp. What we built, why 2-bit matters, the measured numbers and their scope, what we lose, and how to try it.Small-State Coset Trellis Coding at IQ2_XXS Size: An Equal-Bytes Study in llama.cpptag:genrius-ai.github.io,2026-10-24:pingala8-llm/paper.html2026-10-24T17:28:54Z2026-10-24T17:28:54ZPaper: an 8-state trellis code stored in a llama.cpp GGUF file at IQ2_XXS size. Qwen3-4B in 1.24 GB reaches WikiText-2 perplexity 16.64 against 28.61 for IQ2_XXS. Abstract, PDF and BibTeX.How much VRAM do you need to run an LLM locally?tag:genrius-ai.github.io,2026-10-24:pingala8-llm/how-much-vram-for-local-llm.html2026-10-24T15:59:05Z2026-10-24T17:28:54ZA plain-English way to work out GPU memory for a local LLM: file size, KV cache and headroom, with measured Qwen3-4B file sizes from 8.05 GB down to 1.24 GB.IQ2_XXS vs IQ2_KT vs Q2_K vs Q4_K_M: which GGUF quant?tag:genrius-ai.github.io,2026-10-24:pingala8-llm/iq2-xxs-vs-iq2-kt-vs-q2-k-vs-q4-k-m.html2026-10-24T15:59:05Z2026-10-24T17:28:54ZGGUF quant names decoded, with measured Qwen3-4B size, perplexity, quiz scores and speed for Q4_K_M, Q2_K, IQ2_XS, IQ2_XXS, IQ2_KT and IQ1_M.What is a 2-bit LLM? Is 2-bit quantization usable?tag:genrius-ai.github.io,2026-10-24:pingala8-llm/is-2-bit-quantization-usable.html2026-10-24T15:59:05Z2026-10-24T17:28:54ZWhat 2-bit quantization does to an LLM, measured on Qwen3-4B: size, perplexity and quiz scores from IQ1_M to Q2_K, and when 2-bit is a poor fit.How to run Qwen3-4B on a 4 GB or 6 GB GPUtag:genrius-ai.github.io,2026-10-24:pingala8-llm/run-qwen3-4b-on-4gb-6gb-gpu.html2026-10-24T15:59:05Z2026-10-24T17:28:54ZWhich Qwen3-4B file fits a 4 GB or 6 GB graphics card, how much context you keep, and the llama.cpp settings that help, with measured sizes and scores.What is perplexity? LLM perplexity in plain Englishtag:genrius-ai.github.io,2026-10-24:pingala8-llm/what-is-perplexity-llm.html2026-10-24T15:59:05Z2026-10-24T17:28:54ZWhat LLM perplexity measures, what a good score is, and how to compare quantized models fairly, with WikiText-2 numbers for Qwen3-4B from 12.46 to 57.50.How to run Llama 3.1 8B on a 4 GB or 6 GB GPUtag:genrius-ai.github.io,2026-10-24:pingala8-llm/run-llama-3-1-8b-on-4gb-6gb-gpu.html2026-10-24T17:53:59Z2026-10-24T17:53:59ZWhich Llama 3.1 8B file fits a 4 GB or 6 GB graphics card, how much context you keep, and the llama.cpp flags that help, with 2-bit files at 2.39 GB.What is trellis-coded quantization? TCQ, QTIP and IQ2_KT explainedtag:genrius-ai.github.io,2026-10-24:pingala8-llm/what-is-trellis-coded-quantization.html2026-10-24T17:53:59Z2026-10-24T17:53:59ZTrellis-coded quantization (TCQ) in plain English: where it came from, how QTIP, EXL3 and IQ2_KT use it for LLM weights, and what a small trellis buys.What is Hadamard rotation in LLM quantization? QuIP, QuaRot and incoherence explainedtag:genrius-ai.github.io,2026-10-24:pingala8-llm/what-is-hadamard-rotation-quantization.html2026-10-24T17:53:59Z2026-10-24T17:53:59ZWhy LLM quantizers rotate weights with a Hadamard transform before rounding: outliers, incoherence, QuIP, QuIP#, QuaRot and SpinQuant in plain English.