quick_start/v1/05_OpenAI_parameters.ipynb (474 lines of code) (raw):
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# OpenAI Parameters"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Overview\n",
"When making requests to OpenAI models, several parameters can be used to control the behavior and output of the model. \\\n",
"Understanding these parameters helps in fine-tuning the responses to meet specific requirements, whether for generating text, answering questions, or any other use case.\n",
"\n",
"For more detailed examples, refer to the official documentation [Azure OpenAI Service](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference)\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import re\n",
"import requests\n",
"import sys\n",
"import os\n",
"from openai import AzureOpenAI\n",
"import tiktoken\n",
"from dotenv import load_dotenv\n",
"load_dotenv()\n",
"\n",
"client = AzureOpenAI(\n",
" azure_endpoint = os.getenv(\"AZURE_OPENAI_ENDPOINT\"), \n",
" api_key=os.getenv(\"AZURE_OPENAI_KEY\"), \n",
" api_version=\"2024-02-15-preview\"\n",
")\n",
"\n",
"CHAT_COMPLETIONS_MODEL = os.getenv('AZURE_OPENAI_DEPLOYMENT_NAME')\n",
"SEED = 123"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Parameter: max_tokens\n",
"**Description**: The maximum number of tokens to generate in the completion. \\\n",
"**Default**: 16 \\\n",
"**Example**: max_tokens=50\n",
"\n",
"The token count of your prompt plus max_tokens can't exceed the model's context length. \\\n",
"Most models have a context length of 2048 tokens (except for the newest models, which support 4096). Please refer to documentation."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Max Tokens: 16\n",
"\n",
"The best pet really depends on various factors, including your lifestyle, living situation,\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
"Max Tokens: 32\n",
"\n",
"The best pet really depends on individual preferences and lifestyles. Here are some considerations for various popular pets:\n",
"\n",
"1. **Dogs**: If you want a loyal and\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
"Max Tokens: 60\n",
"\n",
"The \"best\" pet can vary greatly depending on individual preferences, lifestyle, living conditions, and what a person is looking for in a pet. Here are some popular choices:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active people or families. They require regular\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
"Max Tokens: 100\n",
"\n",
"The \"best\" pet can vary greatly depending on an individual's lifestyle, preferences, and living situation. Here are some popular options:\n",
"\n",
"1. **Dog**: Known for their loyalty and companionship. They require regular exercise and training but can be great for active individuals or families.\n",
"\n",
"2. **Cat**: More independent than dogs, cats can be affectionate and require less space. They suit people who prefer a lower-maintenance pet.\n",
"\n",
"3. **Fish**: An aquarium can bring beauty and tranquility to\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n"
]
}
],
"source": [
"def call_openai_with_max_tokens(max_tokens):\n",
" response = client.chat.completions.create(\n",
" model=CHAT_COMPLETIONS_MODEL,\n",
" messages = [{\"role\":\"system\", \"content\":\"You are a helpful assistant.\"},\n",
" {\"role\":\"user\",\"content\": \"The best pet is a\"}],\n",
" max_tokens=max_tokens\n",
" )\n",
" return response.choices[0].message.content\n",
"\n",
"# Generate with different presence_penalty values\n",
"penalties = [16, 32, 60, 100]\n",
"for penalty in penalties:\n",
" print(f\"Max Tokens: {penalty}\\n\")\n",
" print(call_openai_with_max_tokens(penalty))\n",
" print(\"\\n\" + \"-\"*80 + \"\\n\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Parameter: temperature\n",
"\n",
"**Description**: Controls the randomness of the output. Lower values make the output more deterministic, while higher values increase randomness. \\\n",
"**Value Range**: 0 to 1 \\\n",
"**Default Value**: 1 \\\n",
"**Example**: temperature=0.7\n",
"\n",
"Higher values means the model will take more risks. \\\n",
"Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer.\n",
"\n",
"---\n",
"**NOTE**: We generally recommend altering this or top_p but not both.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"def call_openai(num_times, prompt, temperature=0.75, use_seed=False):\n",
" for i in range(num_times):\n",
" if use_seed:\n",
" response = client.chat.completions.create(\n",
" model=CHAT_COMPLETIONS_MODEL,\n",
" messages = [{\"role\":\"system\", \"content\":\"You are a helpful assistant.\"},\n",
" {\"role\":\"user\",\"content\": prompt}],\n",
" max_tokens=60,\n",
" seed=SEED,\n",
" temperature = temperature\n",
" )\n",
" else:\n",
" response = client.chat.completions.create(\n",
" model=CHAT_COMPLETIONS_MODEL,\n",
" messages = [{\"role\":\"system\", \"content\":\"You are a helpful assistant.\"},\n",
" {\"role\":\"user\",\"content\": prompt}],\n",
" max_tokens=60,\n",
" temperature = temperature\n",
" )\n",
" print(response.choices[0].message.content)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The best pet really depends on your personal preferences, lifestyle, and living situation. Here are a few popular options and what they might offer:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship. They can be great for active individuals or families, as they often need exercise and enjoy spending time\n",
"The best pet really depends on your personal preferences, lifestyle, and living situation. Here are some common options, each with its own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families who have time for walking, training, and playing.\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options:\n",
"\n",
"1. **Dogs**: Great for companionship, loyalty, and active households. They require regular exercise and attention.\n",
"2. **Cats**: Independent and low-maint\n",
"The best pet really depends on your lifestyle, preferences, and living situation. Here are a few options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require daily exercise and attention.\n",
"\n",
"2. **Cats\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are some popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active people who can spend time training and exercising with them\n",
"The best pet really depends on your personal preferences, lifestyle, and living situation. Here are some common options, each with their own advantages:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship. They can be good for active individuals or families, as they often require exercise and interaction.\n",
"\n",
"2\n",
"The \"best\" pet can vary greatly depending on an individual's lifestyle, preferences, and needs. Here are some popular options, each with their own benefits:\n",
"\n",
"1. **Dog**: Known for their loyalty and companionship, dogs are great for people who have time for regular walks, play, and care\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, along with some considerations for each:\n",
"\n",
"1. **Dog**: Dogs are known for their loyalty and companionship. They can be great for active people who have the time\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are some options based on different needs and environments:\n",
"\n",
"1. **Dogs**: Great for active people who enjoy outdoor activities and want a loyal, interactive companion. Different breeds have different needs and temper\n",
"The best pet depends on your personal preferences, lifestyle, and living situation. Here are some popular options and their benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship. They require regular exercise and attention, making them great for active individuals or families.\n",
"\n",
"2. **Cats**: Independent\n"
]
}
],
"source": [
"# Without seed and temperature, the response is different each time\n",
"call_openai(10, 'The best pet is a ')"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n",
"The best pet really depends on your lifestyle, preferences, and what you're looking for in a companion. Here are a few popular options, each with their own benefits:\n",
"\n",
"1. **Dogs**: Known for their loyalty and companionship, dogs are great for active individuals or families. They require regular exercise and\n"
]
}
],
"source": [
"# Now using a seed and 0 temperature, the response is the much more consisitent\n",
"call_openai(10, 'The best pet is a ', temperature = 0, use_seed=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Parameter: n\n",
"**Description**: Specifies the number of completions to generate for each prompt. \\\n",
"**Default Value**: 1 \\\n",
"**Example**: n = 3 \n",
"\n",
"---\n",
"**Note**: Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for max_tokens and stop."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 The best pet for you depends on various factors, including your lifestyle, living situation, personal preferences, and the amount of time and resources you can dedicate to a pet. Here are some considerations for popular pets:\n",
"\n",
"1. **Dogs**: Great companions for active people who enjoy outdoor activities and can commit\n",
"1 The best pet really depends on your personal preferences, lifestyle, and living situation. Here are a few popular options based on different needs and preferences:\n",
"\n",
"1. **Dogs**: Often considered loyal and great companions. They can be very social and require regular exercise and attention. They come in a variety of\n"
]
}
],
"source": [
"response = client.chat.completions.create(\n",
" model=CHAT_COMPLETIONS_MODEL,\n",
" messages = [{\"role\":\"system\", \"content\":\"You are a helpful assistant.\"},\n",
" {\"role\":\"user\",\"content\": \"The best pet is a \"}],\n",
" max_tokens=60,\n",
" n=2\n",
" )\n",
"\n",
"for index, c in enumerate(response.choices):\n",
" print(index, c.message.content)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Parameter: presence_penalty\n",
"**Description**: Penalizes new tokens based on whether they appear in the text so far, encouraging the model to use new tokens. \\\n",
"**Value Range**: -2.0 to 2.0 \\\n",
"**Default Value**: 0 \\\n",
"**Example**: presence_penalty=0.5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Presence Penalty: 0\n",
"\n",
"Choosing the best pet depends on your lifestyle, preferences, and needs. Here are a few options based on different criteria:\n",
"\n",
"1. **Dogs**: Great for companionship and active lifestyles. They're loyal and require regular exercise.\n",
"2. **Cats**: More independent than dogs, usually require less maintenance,\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
"Presence Penalty: 0.5\n",
"\n",
"The best pet can vary depending on an individual's lifestyle, preferences, and needs. Here are a few considerations for different types of pets:\n",
"\n",
"1. **Dogs:** Great companions, highly social, and vary widely in size, temperament, and activity level. Ideal for those who enjoy outdoor activities and have time\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
"Presence Penalty: 1.0\n",
"\n",
"The \"best\" pet depends on various factors, including your lifestyle, living situation, and personal preferences. Different animals can make great pets for different people:\n",
"\n",
"1. **Dogs** - Known for their loyalty and companionship, they're great if you have time for daily walks and playtime.\n",
" \n",
"2.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
"Presence Penalty: 1.5\n",
"\n",
"The best pet really depends on your personal preferences, lifestyle, and what you're looking for in a companion. Here are some common choices:\n",
"\n",
"- **Dogs**: Known for their loyalty and companionship. They require regular exercise and attention.\n",
"\n",
"- **Cats**: Independent but can be very affectionate. They tend\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
"Presence Penalty: 2.0\n",
"\n",
"The best pet can vary greatly depending on individual circumstances and preferences. Here are a few factors to consider when determining the best pet for you:\n",
"\n",
"1. **Lifestyle:** If you have a busy lifestyle, a low-maintenance pet like fish or a reptile might be suitable. For those who enjoy physical\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n"
]
}
],
"source": [
"def call_openai_with_presence_penalty(presence_penalty):\n",
" response = client.chat.completions.create(\n",
" model=CHAT_COMPLETIONS_MODEL,\n",
" messages = [{\"role\":\"system\", \"content\":\"You are a helpful assistant.\"},\n",
" {\"role\":\"user\",\"content\": \"The best pet is a\"}],\n",
" max_tokens=60,\n",
" presence_penalty=presence_penalty\n",
" )\n",
" return response.choices[0].message.content\n",
"\n",
"# Generate with different presence_penalty values\n",
"penalties = [0, 0.5, 1.0, 1.5, 2.0]\n",
"for penalty in penalties:\n",
" print(f\"Presence Penalty: {penalty}\\n\")\n",
" print(call_openai_with_presence_penalty(penalty))\n",
" print(\"\\n\" + \"-\"*80 + \"\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Parameter: frequency_penalty\n",
"**Description**: Penalizes new tokens based on their existing frequency in the text so far, reducing the likelihood of repeating the same line verbatim. \\\n",
"**Value Range**: -2.0 to 2.0 \\\n",
"**Default Value**: 0 \\\n",
"**Example**: frequency_penalty=0.5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Use cases to explore\n",
"1. **Compare Responses** \\\n",
"Generate multiple completions to compare and choose the best response for your use case.\n",
"\n",
"2. **Increase Diversity** \\\n",
"Use multiple completions to get a variety of responses, which is useful in creative applications.\n",
"\n",
"3. **Enhance Robustness** \\\n",
"Generate multiple responses to ensure consistency and accuracy across different completions.\n",
"\n",
"#### Best Practices\n",
"1. **Optimize Prompt Length** \\\n",
"Keep your prompts concise but informative to ensure the model has enough context.\n",
"\n",
"2. **Adjust Temperature and Top_p** \\\n",
"Use these parameters to balance between deterministic and creative responses.\n",
"\n",
"3. **Monitor Token Usage** \\\n",
"Be mindful of the max_tokens parameter to manage costs and response length.\n",
"\n",
"4. **Use Stopping Sequences** \\\n",
"Define stopping sequences to control where the model should stop generating text, ensuring the output is within the desired context.\n",
"\n",
"5. **Generate Multiple Completions** \\\n",
"Use the n parameter to generate multiple completions and select the best one for your needs."
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.5"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}