Loading Larkspur’s toy experts model…

Larkspur’s toy runs 2 of 8 rooms a floor; DeepSeek-V3 runs 8 plus 1 shared of 256 per layer.

Chapter 7 of 7

Workbench: a shape for the server

Larkspur’s server has a memory budget and a speed budget. The team has 4 published model cards to choose from.

This chapter: Which model and precision fit both budgets with the most weights?

About this chapter

You will learn

Check total weights against memory
Check active weights against the work per token
Why precision helps memory, not work

Words you’ll meet

Budget: A limit Larkspur sets: memory for weights, and weights at work per token. Planning numbers.
Precision: Bits per weight. Fewer bits shrink memory (Inference’s quantization); the work per token stays.
Workbench: The chapter’s free-play bench: change the plan and see every result.

Step 1 of 2

Larkspur’s : 160 GB of memory, 40B weights at work per token. sets bytes per weight.

Fit both with the most weights.

671B total, 16-bit OVER BUDGET

Model cardMixtral 8x7B: 46.7B total, 12.9B activeMixtral 8x22B: 141B total, 39B activeDeepSeek-V3: 671B total, 37B activeLlama 3.1 70B: 70B total, 70B active

Precision16-bit (2 bytes a weight)8-bit (1 byte a weight)4-bit (0.5 bytes a weight)

DeepSeek-V3 at 16 bits on Larkspur’s server
Budget	This plan
Memory (budget 160 GB)	1342 GB
Work per token (budget 40B)	37B active
Larkspur’s toy, beside it	68,672 total, 43,328 active, 137,344 bytes

Budgets are Larkspur’s planning numbers: 160 GB of GPU memory for weights (two 80 GB cards); at most 40B weights at work per word, to keep replies fast.

DeepSeek-V3’s report (2024) describes a mixture of experts with 671B total parameters, 37B activated for each token: 256 routed experts and 1 shared per layer, 8 routed chosen per token. arxiv.org · checked 2026-10-25