Tinker Load balancing

by Samy (Kondasamy Jayaraman) · drawn with Colorfigures by April Zhu

No run yet.

Ash alone, before any request.Ash 0 waiting

Chapter 1 of 7

One server’s ceiling

Larkspur’s shop runs on one server, Ash. At one second in, a sale starts and requests arrive twice as fast.

This chapter: What happens to the wait when requests arrive faster than one server can answer?

About this chapter

You will learn

  • A server answers only so many requests a second
  • Past that, a queue grows and every wait with it
  • Why p99 shows what the slowest customers feel

Words you’ll meet

Capacity
The most requests a server can answer per second before a queue forms.
Queue
Requests waiting for a free slot on a server.
p99
The wait 99 of every 100 requests beat: the slow edge of what customers feel.
Load balancer
The switch in front of several servers that picks which one gets each request.

Step 1 of 5

Ash works on 4 requests at once. The board shows every wait; marks the slowest edge.

Run the morning.