Exam-style
An extraction job pulls fixed fields out of invoices into a JSON schema. Run the same invoice twice and the field wording differs, and occasionally the model appends a sentence of commentary after the JSON. Which inference settings should change, and in which direction?
Reveal the answer
C. Lower temperature and top-p toward 0, and set max tokens to bound the response length
Temperature and top-p both control how much randomness is allowed into token selection, and structured extraction wants as little as possible, so both go toward 0; max tokens then caps length and cost. Raising any of the sampling parameters does the opposite and widens the variation being complained about. Cutting max tokens alone truncates the output mid-JSON rather than stopping the commentary, and a larger model at default settings is still sampling randomly.
Q. Outputs are too random for a structured extraction task. Which inference parameters, and which way?
A. Lower temperature and top-p toward 0 for deterministic, focused output; raise them for creative variety. Set max tokens to bound length and cost.
Why? Temperature and top-p control randomness; extraction wants low, brainstorming wants higher.