Docs menu
How Rodatacenter works
The server behind the chat, why a CPU-only workstation can run these models, and what each error means.
The chat site is llama.cpp’s own web interface. Each message sends the whole conversation to the server, which runs it through the selected model on the workstation’s CPU and streams the answer back. The workstation is reached through a Cloudflare Tunnel, so nothing in the lab is open to the internet. A filter in front of the model server strips tool definitions, blocks model downloads and fills in the model name when a request leaves it out.
The workstation has a quad-core Intel i7-6700, 12 GB of RAM and no usable GPU. It was set up in October 2026.
Why a CPU is fast enough
A reply has two phases. Reading the prompt processes the conversation in batches, so arithmetic limits it, at 40 to 150 tokens per second depending on the model. Writing the answer produces one token at a time, and each token reads the model’s active weights from memory again, so memory bandwidth sets the speed. That is why the 4B dense model writes about 6 tokens per second while the 8B mixture-of-experts model writes about 18: the mixture-of-experts model reads only the roughly 1B parameters its router selects for each token.
The weights are stored at about 4.5 bits each instead of 16, which makes them about 3.5 times smaller and writing about that much faster, at a small cost in accuracy. Both model families are hybrid designs in which most layers keep a fixed-size state, so a 32k-token conversation needs hundreds of megabytes of memory rather than gigabytes.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| The site does not load, or Cloudflare error 1033 | The workstation is off, offline or restarting | Try again later; if it lasts, mention @it-services in Discord |
| 502 “model server is starting or down” | The model server is restarting | Wait a minute and retry |
| The first reply takes 10 to 40 seconds | A model switch is loading weights | Expected after a switch |
| A long wait before any words | A long conversation or pasted document is being read | Start a new chat for a new topic |
| Context exceeded error | The chat passed 32,768 tokens | Start a new chat |
| API: 403 with error 1010 | The client’s User-Agent was blocked | Set any User-Agent header |
| API: connection cut after about 100 seconds | The request was not streamed | Use "stream": true |