SFT (Supervised Fine-Tuning)

Definition

Supervised fine-tuning is the step in which a pretrained model is trained further on curated prompt-and-response pairs so that it imitates demonstrated behaviour. The objective is the same next-token prediction used in pretraining; what changes is the data. Instead of undifferentiated text scraped at scale, the model sees a comparatively small set of examples that show the behaviour wanted: the format of a good answer, the reasoning laid out step by step, the refusal where a refusal is correct, the tool call where a tool call is correct. It is normally the first post-training stage, and it is what makes a base model usable as an assistant.

It is imitation learning, and imitation is literal. The model inherits the strengths of the demonstrations and also their errors, their habits and their blind spots. A demonstration with a plausible-but-wrong intermediate step teaches the model to produce plausible-but-wrong intermediate steps. Because these sets are small relative to pretraining, each example carries weight, and a handful of bad examples in a specialist domain can do more damage than their number suggests. Quality and coverage dominate volume here in a way that is unusual in machine learning.

That makes authoring the standard bottleneck. Writing a correct expert-level demonstration — a full clinical rationale, a sound proof, a defensible contract analysis, a working piece of secure code — takes someone qualified to produce it, and it takes longer than judging one. Serious programmes therefore run written guidelines, reference answers with known-correct content, tiered review and arbitration for disagreements, and they measure the authors rather than trusting the credential. Instruction tuning is the best-known form of this stage; reasoning traces and tool-use demonstrations are the forms that most often need genuine domain expertise.

How DeepenSkill approaches it

Authoring demonstrations at this level is expert work, and DeepenSkill supplies the experts rather than a finished catalogue. Everyone clears the same bar before starting: credentials verified, a standardized domain test passed, then a practical task scored against gold reference data. On live work the team measuring the experts is separate from the team running them — the platform rejects a validation report authored by the party that delivered the work — agreement statistics are reported with the method named, the item and rater counts stated and a 95% interval attached, and every judgment traces to a verified contributor, a task spec and a timestamp. See expert SFT work, the broader annotation and judgment work around it, and a sample verification bundle you can recompute yourself.

FAQ

• What is supervised fine-tuning?

  Training a pretrained model further on curated prompt-and-response pairs so it imitates the behaviour those examples demonstrate — format, reasoning, refusals and tool use included.

• How is SFT different from RLHF?

  SFT teaches a model by showing it the answer to imitate. RLHF teaches it by having people judge which of its own answers is better and optimizing against those preferences. SFT normally comes first.

• Why does SFT data quality matter more than quantity?

  Because the sets are small relative to pretraining, so each example carries weight. A demonstration with a plausible-but-wrong step teaches the model to produce plausible-but-wrong steps.