servermarket

Blog · · prices update live

Renting a GPU server for open LLMs: how much VRAM you need and what it costs

A rule of thumb for VRAM per model size and quantization, and live monthly prices for in-stock GPU servers with enough memory.

As of , the cheapest in-stock GPU server is the Intel Core i7-7700 from Hetzner at €87.00/mo in Falkenstein. Next: the Intel Core i7-7700 from Hetzner at €87.00/mo in Nuremberg. 112 matching servers are in stock across all providers (prices excl. VAT).

How much VRAM a model needs

To serve a model, its weights have to fit in GPU memory, plus room for the KV cache, which grows with context length and the number of parallel requests. A workable rule of thumb:

  • 16-bit (FP16/BF16): about 2 GB of VRAM per billion parameters.
  • 8-bit (FP8/INT8): about 1 GB per billion parameters.
  • 4-bit (GPTQ, AWQ, GGUF Q4): about 0.5–0.6 GB per billion parameters.
  • Add 20–30% on top for the KV cache and runtime at moderate context lengths, more for long contexts or many concurrent users.

So an 8B model at 16-bit fits on a 24 GB card, a 32B model at 4-bit fits in about 24 GB, and a 70B model needs roughly 40–48 GB at 4-bit or about 160 GB at 16-bit, which means several GPUs or an 80 GB+ card.

Cheapest GPU servers by price per GPU

In-stock GPU servers, cheapest per GPU

Prices checked excl. VAT

See all 112

Data-center GPUs for large models

For 70B-class models at higher precision, or for high-throughput serving, look at H100, H200 and L40S servers.

NVIDIA H100 servers

Prices checked excl. VAT

See all 7
NVIDIA L40S servers

Prices checked excl. VAT

See all 6

Monthly dedicated or hourly cloud?

Hourly GPU clouds are the better deal for experiments and bursty jobs. If a GPU runs most of the day, every day, a monthly dedicated GPU server usually costs far less than the same hours on demand, and nobody else shares the card. Ask the deployment planner for a mix, or let your AI query the live list.