Tinker Load balancing

by Samy (Kondasamy Jayaraman) · drawn with Colorfigures by April Zhu

No run yet.

Least outstanding, Dogwood slow but answering.Ash 0 waiting · Birch 0 waiting · Cedar 0 waiting · Dogwood 0 waiting

Chapter 6 of 7

The slow tail

Dogwood is slow but answers its checks, so it stays in. Least outstanding sends it little, yet customers who land there still wait.

This chapter: Why does one slow server set the slowest customers’ wait, and how do you cut it?

About this chapter

You will learn

  • p50 and p99 tell different stories
  • A hedged request asks a second server after a wait
  • Hedge too early and the copies crowd the servers

Words you’ll meet

p50
The median wait: half of requests are faster.
Tail
The slowest few requests, the ones p99 measures.
Hedged request
A second copy of a slow request sent to another server; the first answer wins.

Step 1 of 5

Most requests land on fine servers. The few on Dogwood take five times as long: they are the .

Run the sale.