Understanding AI Parameters

The few dials that make a difference

Think of parameters like mood and length controls:

• Temperature: how adventurous the wording is. Lower (0.0–0.3) = careful and consistent. Higher (1.0+) = creative and surprising.
• Top‑p: another way to control adventurousness. Use this instead of temperature when you prefer to say “only pick from the top X% most likely words.” Pick one: temperature or top‑p, not both.
• Max tokens: how long the model can talk. Short caps keep answers tight; bigger caps allow detail but cost more.

Two nudges I reach for when needed:

• Frequency penalty: reduce repetition in longer responses.
• Presence penalty: encourage the model to introduce new ideas.

A few more handy switches

• Frequency penalty (frequency_penalty): discourages repeating the same tokens. Increase to reduce repetition in long outputs.
• Presence penalty (presence_penalty): encourages introducing new tokens/topics. Increase to promote novelty and avoid sticking to one idea.
• Top‑k (top_k): limit choices to the top K most likely next tokens. Lower K = safer and more focused; higher K = more variety. Often leave unset and prefer temperature or top_p unless you need strict caps.

Quick recipes

• Precise answer: (temperature=0.1–0.2, max_tokens=120)
• Friendly explanation: (temperature=0.5–0.7, max_tokens=200–300)
• Brainstorming: (temperature=0.9 or top_p 0.95, presence_penalty=0.4–0.8)
• Reduce repetition in long text: (frequency_penalty=0.5–0.9, temperature=0.3–0.6)
• Deterministic code: (temperature=0.0–0.2), consider a conservative (max_tokens)

One simple example

import OpenAI from "openai";

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const completion = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [
    { role: "system", content: "You are a concise, friendly teacher." },
    { role: "user", content: "Explain embeddings like I'm new to ML." }
  ],
  temperature: 0.6,
  presence_penalty: 0.4,
  max_tokens: 220
});

console.log(completion.choices[0].message.content);

Try it live

This example features an AI with very high temperature, so very random responses are common.

Key Takeaways:

• Temperature and top‑p control creativity-pick one.
• Max tokens caps length; penalties adjust repetition and novelty.
• Use presets for precise, friendly, brainstorming, or code answers.

More Resources:

• Prompts 101
• Prompt Engineering Techniques

Sources:

• OpenAI API Reference - Chat Completions: https://platform.openai.com/docs/api-reference/chat/create
• OpenAI Documentation - Parameter Guidelines and Best Practices: https://platform.openai.com/docs/guides/text-generation