logprobs is a mode where LLMs return the probability of each token. topLogProbs also returns list list of alternate tokens
and their log probabilities. This can be useful for debugging and understanding the model’s behavior.
When logprobs is enabled, the cli will color the output based on the probability of each token.
Blue color indicates high probability and red color indicates low probability.
Here is an example of logprobs in action when running a poem prompt with gpt-4o.
In the whisper of trees, the night softly speaks, Where the moonlight weaves through the shadows it seeks. Stars twinkle above, like dreams far away, Painting the night with the dawn’s gentle sway.
Top logprobs
You can enable top-logprobs in the following ways:
Use the top-logprobs flag on the run command. It enables logprobs as well.
Terminal window
npxgenaiscriptrun...--top-logprobs4
add the topLogprobs flag to the script metadata
script({ topLogProbs: 4, ...})
Colored output
When top-logprobs are enabled, the console window is colored with the entropy of the alternate tokens.
In the whisper of trees, the night softly speaks, Where the moonlight weaves through the shadows it seeks. Stars twinkle above, like dreams far away, Painting the night with the dawn’s gentle sway.
Alternate tokens
The trace contains a rendering of the alternate tokens with colored output based on the logprobs.
