Pingala-8 guides

Qwen3.8 27B VRAM requirements: which GPU do you need?

Genrius AI guide. October 25, 2026, updated November 1, 2026. About 8 minutes.

Will it fit your GPU? Normal vs Pingala-8 calculator

Qwen3.8-27B is the dense 27B model in Qwen's 2026 line. At 27 billion parameters it sits right at the edge of a single consumer GPU, so the usual question is simple: which card, at which quantization, with how much context? This page answers it with arithmetic from the model's own config file, Qwen/Qwen3.8-27B config.json @ 1d4bf0f, using the same formulas as our GPU fit calculator.

Memory per quantization

Two things take most of the memory: the weights, which depend on the quantization, and the KV cache, which grows with the context. Here are both for the four formats people ask about most.

Memory estimates from public configs, not measurements: we have not run this model. Weights = parameters x bits per weight / 8, with llama.cpp's whole-file bits per weight (FP16 16.0005, Q8_0 8.5008, Q4_K_M 4.8944, IQ2_XXS 2.3824, from the Llama-3.1-8B table in llama.cpp's tools/quantize/README.md). KV cache at 16-bit, context padded to 256 cells as llama.cpp does. GB = 10⁹ bytes. Add about 1.5 GB for the runtime's compute buffers and your desktop. These are the calculator's formulas.

As a check on the estimate, unsloth's published files (unsloth/Qwen3.8-27B-GGUF) are 29.05 GB for Q8_0, 16.46 GB for UD-Q4_K_M and 7.27 GB for UD-IQ2_XXS. They are a little smaller than our rows because the parameter count above includes the vision encoder and the multi-token-prediction layer, which a text GGUF leaves out or ships separately (the vision projector, mmproj, is 0.93 GB at BF16), and because unsloth's "UD" files mix quant types per tensor.

Which GPU fits at 4k, 8k and 32k context

The largest of the four formats in the table above whose weights plus 16-bit KV cache leave at least 1.5 GB free on the card; "tight" means it fits but with less than 1.5 GB left. Card memory from NVIDIA's spec pages, taken at face value. On paper, not tested on these cards; the one measured run is our 8.31 GB Pingala-8 file on a 12 GB RTX 5070 Ti Laptop GPU, above.

The pattern is simple. On 24 GB, Q4_K_M fits at every context in the table, and at 32k tokens about 4.7 GB is still free before working buffers. Q8_0 at about 29.5 GB needs a 32 GB RTX 5090 and is tight there at 32k. On a 16 GB card, 17 GB of Q4_K_M weights alone do not fit, so you either drop to a 2-bit file or keep some layers in system RAM with llama.cpp's -ngl, which is much slower. On 12 GB only a 2-bit file fits, and it gets tight at 32k. On 8 GB even the 2-bit file is larger than the card.

Why the KV cache is so small: hybrid attention

Qwen3.8-27B is a hybrid. Its config lists 64 layers in layer_types: 48 are linear-attention layers and 16 are full-attention layers (one in every four, full_attention_interval = 4). Only the full-attention layers keep a cache that grows with every token. Each has 4 key-value heads of size 256, so the cache costs 2 x 4 x 256 x 2 bytes = 4,096 bytes per token per layer, or 65,536 bytes per token for all 16.

The linear-attention layers keep a fixed state instead, the same size at 100 tokens or 100,000. llama.cpp stores it in 32-bit floats (src/llama-hparams.cpp, n_embd_r and n_embd_s): per layer, (conv kernel 4 - 1) x (2 x 16 key heads x 128 + 48 value heads x 128) for the convolution plus 128 x 48 x 128 for the recurrent state. Over 48 layers that is 156,893,184 bytes, 0.16 GB, included in every KV number on this page.

The result: 0.69 GB at 8,192 tokens, 8.75 GB at 131,072 and 17.34 GB at the full 262,144-token window. A non-hybrid 64-layer model with 8 key-value heads of size 128, the shape of Qwen3-32B, needs 262,144 bytes per token, four times as much. For long context on one card, that difference matters more than the quantization.

Settings that help

With llama.cpp, -ngl 99 keeps every layer on the GPU and -c sets the context. -fa on -ctk q8_0 -ctv q8_0 stores the full-attention cache at 8 bits, which roughly halves that part (the fixed state stays 32-bit). Start with the context you really need: at 32k the cache is only 2.30 GB, so on a 24 GB card Q4_K_M is the safe default.

Pingala-8 and this model

Pingala-8 is a 2-bit format at IQ2_XXS size. The Qwen3.8-27B build is released: Genrius-AI/Qwen3.8-27B-Pingala8-2bit, version 2, 8.31 GB, measured in the box at the top. Version 2 keeps the trellis coder of version 1 and changes how bits are spread across tensors, which took perplexity from 7.34 to 6.89 at the same size. It is ahead of llama.cpp IQ2_XXS of the same model, roughly level with unsloth's UD-IQ2_S, and behind DASH-Q IQ2_XS.

The file runs only in our patched llama.cpp (CUDA); stock llama.cpp, Ollama and LM Studio cannot load it. On other models: on Qwen3-4B, Pingala-8 is 1.24 GB with a WikiText-2 perplexity of 16.64 against 28.61 for IQ2_XXS at the same size; on Qwen3-14B it loses narrowly to IQ2_KT (9.57 against 9.40). Details are in the paper, DOI 10.5281/zenodo.23481757 (the Qwen3.8-27B numbers are new and not in the paper), and the benchmark dataset.

Questions people ask

How much VRAM does Qwen3.8-27B need?

On paper about 55.6 GB of weights at FP16, 29.5 GB at Q8_0, 17.0 GB at Q4_K_M and 8.3 GB at IQ2_XXS, plus a KV cache of 0.69 GB at 8,192 tokens and about 1.5 GB of working buffers. These are estimates from the public config, not measurements.

Can Qwen3.8-27B run on a 16 GB GPU?

Not at Q4_K_M fully on the GPU: the weights alone are about 17 GB. A 2-bit file of about 8.3 GB fits with a 32k context on paper, or you can keep some layers in system RAM, which is much slower.

Can an RTX 4090 run Qwen3.8-27B?

Yes, on paper. Q4_K_M weights of about 17 GB plus a 32,768-token cache of 2.3 GB leave about 4.7 GB of the 24 GB for working buffers.

Is there a Pingala-8 version of Qwen3.8-27B?

Yes, since November 1, 2026: Genrius-AI/Qwen3.8-27B-Pingala8-2bit on Hugging Face, version 2, 8.31 GB. Its WikiText-2 perplexity is 6.89, against 7.88 for llama.cpp IQ2_XXS (8.43 GB, same imatrix) and 7.34 for our version 1. The published DASH-Q IQ2_XS file (8.55 GB) is better, at 6.54. It needs the patched Pingala-8 llama.cpp runtime (CUDA); stock llama.cpp, Ollama and LM Studio cannot load it. Model card.

Can Qwen3.8-27B run on a 12 GB GPU?

Yes, measured with a 2-bit file. Our Pingala-8 Qwen3.8-27B file (8.31 GB) runs with every layer on one 12 GB RTX 5070 Ti Laptop GPU (WSL2, -ngl 99) at 37.7 tokens per second token generation, batch 1. Q4_K_M, about 17 GB of weights, does not fit on a 12 GB card.

More guides

• Can you run Qwen3.8-27B on a 12 GB GPU?
• How much VRAM do you need to run an LLM locally?
• What is a 2-bit LLM, and is 2-bit quantization usable?
• IQ2_XXS vs IQ2_KT vs Q2_K vs Q4_K_M: which GGUF quant?
• What is perplexity? LLM perplexity in plain English
• How to run Qwen3-4B on a 4 GB or 6 GB GPU
• How to run Llama 3.1 8B on a 4 GB or 6 GB GPU
• What is trellis-coded quantization? TCQ, QTIP and IQ2_KT
• What is Hadamard rotation in LLM quantization?
• Qwen3.5-2B under 1 GB: 2-bit GGUFs measured
• Gemma 4 VRAM requirements: 26B-A4B and 31B
• Can you run gpt-oss-20b on a 12 GB or 16 GB GPU?
• Qwen3.6 35B-A3B on a consumer GPU: VRAM and MoE offload
• NaN perplexity with IQ1_S, IQ2_S or IQ3_S GGUFs on an RTX 50 laptop GPU