TinkerLoading Larkspur’s toy experts model…
“Questions go to the billing team through”
Chapter 3 of 7
What the experts learned
Training kept sending similar tokens to the same rooms. Larkspur’s team asks what its toy’s rooms ended up good at.
This chapter: Do experts split by topic, or by kind of token?
About this chapter
You will learn
- What the toy’s rooms took most often
- Why experts split by kind of token
- What a large model’s authors found
Words you’ll meet
- Token
- A piece of text a model reads: a word, part of a word, or a mark.
- Specialisation
- What an expert ends up taking most: here, measured over the toy’s training text.
Step 1 of 4
In training, each kept going to the rooms that helped. Over time each room got its .
Predict first.