Measure before you train

The team has three versions of the assistant: prompted, RAG and fine-tuned. Before anyone pays to train again, the release gate from the Evals module gets a say.

This chapter: How do you decide, with numbers, whether fine-tuning beat prompting and documents?

You will learn

Why you set a baseline before you train
How Evals’ release gate compares the three versions question by question
Why a version can sound better and still fail the gate

Words you’ll meet (2)

Baseline: The version a new one is measured against: here, the RAG assistant that already ships.
Release gate: The rule that decides, from a test run, whether a candidate ships.

The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.

Step 1 of 4

Before anyone trains again, measure. A baseline is the version a new one is measured against: here RAG, the assistant Larkspur ships. Run it on the 12 test questions from the Evals module. Its answers are recorded; the key-fact check grades them live.