audio-completions-python.md

manager	author	ms.author	ms.service	ms.topic	ms.date
nitinme	eric-urban	eur	azure-ai-openai	include	1/21/2025

Library source code | Package | Samples

[!INCLUDE Audio completions introduction]

Use this guide to get started generating audio with the Azure OpenAI SDK for Python.

Prerequisites

• An Azure subscription. Create one for free.
• Python 3.8 or later version. We recommend using Python 3.10 or later, but having at least Python 3.8 is required. If you don't have a suitable version of Python installed, you can follow the instructions in the VS Code Python Tutorial for the easiest way of installing Python on your operating system.
• An Azure OpenAI resource created in one of the supported regions. For more information about region availability, see the models and versions documentation.
• Then, you need to deploy a gpt-4o-mini-audio-preview model with your Azure OpenAI resource. For more information, see Create a resource and deploy a model with Azure OpenAI.

Microsoft Entra ID prerequisites

For the recommended keyless authentication with Microsoft Entra ID, you need to:

• Install the Azure CLI used for keyless authentication with Microsoft Entra ID.
• Assign the Cognitive Services User role to your user account. You can assign roles in the Azure portal under Access control (IAM) > Add role assignment.

Set up

1. Create a new folder audio-completions-quickstart and go to the quickstart folder with the following command:

   mkdir audio-completions-quickstart && cd audio-completions-quickstart

2. Create a virtual environment. If you already have Python 3.10 or higher installed, you can create a virtual environment using the following commands:

   Windows

   py -3 -m venv .venv
   .venv\scripts\activate

   Linux

   python3 -m venv .venv
   source .venv/bin/activate

   macOS

   python3 -m venv .venv
   source .venv/bin/activate

   ---

   Activating the Python environment means that when you run python or pip from the command line, you then use the Python interpreter contained in the .venv folder of your application. You can use the deactivate command to exit the python virtual environment, and can later reactivate it when needed.

   [!TIP] We recommend that you create and activate a new Python environment to use to install the packages you need for this tutorial. Don't install packages into your global python installation. You should always use a virtual or conda environment when installing python packages, otherwise you can break your global installation of Python.

3. Install the OpenAI client library for Python with:

   pip install openai

4. For the recommended keyless authentication with Microsoft Entra ID, install the azure-identity package with:

   pip install azure-identity

Retrieve resource information

[!INCLUDE resource authentication]

Generate audio from text input

Microsoft Entra ID

1. Create the to-audio.py file with the following code:

   import requests
   import base64 
   import os 
   from openai import AzureOpenAI
   from azure.identity import DefaultAzureCredential, get_bearer_token_provider
   
   token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
   
   # Set environment variables or edit the corresponding values here.
   endpoint = os.environ['AZURE_OPENAI_ENDPOINT']
   
   # Keyless authentication
   client=AzureOpenAI(
       azure_ad_token_provider=token_provider,
       azure_endpoint=endpoint,
       api_version="2025-01-01-preview"
   )
   
   # Make the audio chat completions request
   completion=client.chat.completions.create(
       model="gpt-4o-mini-audio-preview",
       modalities=["text", "audio"],
       audio={"voice": "alloy", "format": "wav"},
       messages=[
           {
               "role": "user",
               "content": "Is a golden retriever a good family dog?"
           }
       ]
   )
   
   print(completion.choices[0])
   
   # Write the output audio data to a file
   wav_bytes=base64.b64decode(completion.choices[0].message.audio.data)
   with open("dog.wav", "wb") as f:
       f.write(wav_bytes)

2. Run the Python file.

   python to-audio.py

API key

1. Create the to-audio.py file with the following code:

   import base64 
   import os 
   from openai import AzureOpenAI 
   
   # Set environment variables or edit the corresponding values here.
   endpoint = os.environ['AZURE_OPENAI_ENDPOINT']
   api_key = os.environ['AZURE_OPENAI_API_KEY']
   
   client = AzureOpenAI(
       api_version="2025-01-01-preview",  
       api_key=api_key,
       azure_endpoint=endpoint
   )
   
   # Make the audio chat completions request
   completion = client.chat.completions.create(
       model="gpt-4o-mini-audio-preview",
       modalities=["text", "audio"],
       audio={"voice": "alloy", "format": "wav"},
       messages=[
           {
               "role": "user",
               "content": "Is a golden retriever a good family dog?"
           }
       ]
   )
   
   print(completion.choices[0])
   
   # Write the output audio data to a file
   wav_bytes = base64.b64decode(completion.choices[0].message.audio.data)
   with open("dog.wav", "wb") as f:
       f.write(wav_bytes)

2. Run the Python file.

   python to-audio.py

---

Wait a few moments to get the response.

Output for audio generation from text input

The script generates an audio file named dog.wav in the same directory as the script. The audio file contains the spoken response to the prompt, "Is a golden retriever a good family dog?"

Generate audio and text from audio input

Microsoft Entra ID

1. Create the from-audio.py file with the following code:

   import base64
   import os
   from openai import AzureOpenAI
   from azure.identity import DefaultAzureCredential, get_bearer_token_provider
   
   token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
   
   # Set environment variables or edit the corresponding values here.
   endpoint = os.environ['AZURE_OPENAI_ENDPOINT']
   
   # Keyless authentication
   client=AzureOpenAI(
       azure_ad_token_provider=token_provider,
       azure_endpoint=endpoint
       api_version="2025-01-01-preview"
   )
   
   # Read and encode audio file  
   with open('dog.wav', 'rb') as wav_reader: 
       encoded_string = base64.b64encode(wav_reader.read()).decode('utf-8') 
    
   # Make the audio chat completions request
   completion = client.chat.completions.create( 
       model="gpt-4o-mini-audio-preview", 
       modalities=["text", "audio"], 
       audio={"voice": "alloy", "format": "wav"}, 
       messages=[ 
           { 
               "role": "user", 
               "content": [ 
                   {  
                       "type": "text", 
                       "text": "Describe in detail the spoken audio input." 
                   }, 
                   { 
                       "type": "input_audio", 
                       "input_audio": { 
                           "data": encoded_string, 
                           "format": "wav" 
                       } 
                   } 
               ] 
           }, 
       ] 
   ) 
   
   print(completion.choices[0].message.audio.transcript)
   
   # Write the output audio data to a file
   wav_bytes = base64.b64decode(completion.choices[0].message.audio.data)
   with open("analysis.wav", "wb") as f:
       f.write(wav_bytes)

2. Run the Python file.

   python from-audio.py

API key

1. Create the from-audio.py file with the following code:

   import base64
   import os
   from openai import AzureOpenAI
   
   # Set environment variables or edit the corresponding values here.
   endpoint = os.environ['AZURE_OPENAI_ENDPOINT']
   api_key = os.environ['AZURE_OPENAI_API_KEY']
   
   client = AzureOpenAI(
       api_version="2025-01-01-preview",  
       api_key=api_key, 
       azure_endpoint=endpoint
   )
   
   # Read and encode audio file  
   with open('dog.wav', 'rb') as wav_reader: 
       encoded_string = base64.b64encode(wav_reader.read()).decode('utf-8') 
    
   # Make the audio chat completions request
   completion = client.chat.completions.create( 
       model="gpt-4o-mini-audio-preview", 
       modalities=["text", "audio"], 
       audio={"voice": "alloy", "format": "wav"}, 
       messages=[ 
           { 
               "role": "user", 
               "content": [ 
                   {  
                       "type": "text", 
                       "text": "Describe in detail the spoken audio input." 
                   }, 
                   { 
                       "type": "input_audio", 
                       "input_audio": { 
                           "data": encoded_string, 
                           "format": "wav" 
                       } 
                   } 
               ] 
           }, 
       ] 
   ) 
   
   print(completion.choices[0].message.audio.transcript)
   
   # Write the output audio data to a file
   wav_bytes = base64.b64decode(completion.choices[0].message.audio.data)
   with open("analysis.wav", "wb") as f:
       f.write(wav_bytes)

2. Run the Python file.

   python from-audio.py

---

Wait a few moments to get the response.

Output for audio and text generation from audio input

The script generates a transcript of the summary of the spoken audio input. It also generates an audio file named analysis.wav in the same directory as the script. The audio file contains the spoken response to the prompt.

Generate audio and use multi-turn chat completions

Microsoft Entra ID

1. Create the multi-turn.py file with the following code:

   import base64 
   import os 
   from openai import AzureOpenAI 
   from azure.identity import DefaultAzureCredential, get_bearer_token_provider
   
   token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
   
   # Set environment variables or edit the corresponding values here.
   endpoint = os.environ['AZURE_OPENAI_ENDPOINT']
   
   # Keyless authentication
   client=AzureOpenAI(
       azure_ad_token_provider=token_provider,
       azure_endpoint=endpoint,
       api_version="2025-01-01-preview"
   )
   
   # Read and encode audio file  
   with open('dog.wav', 'rb') as wav_reader: 
       encoded_string = base64.b64encode(wav_reader.read()).decode('utf-8') 
   
   # Initialize messages with the first turn's user input 
   messages = [
       { 
           "role": "user", 
           "content": [ 
               { "type": "text", "text": "Describe in detail the spoken audio input." }, 
               { "type": "input_audio", 
                   "input_audio": { 
                       "data": encoded_string, 
                       "format": "wav" 
                   } 
               } 
           ] 
       }] 
   
   # Get the first turn's response
   
   completion = client.chat.completions.create( 
       model="gpt-4o-mini-audio-preview", 
       modalities=["text", "audio"], 
       audio={"voice": "alloy", "format": "wav"}, 
       messages=messages
   ) 
   
   print("Get the first turn's response:")
   print(completion.choices[0].message.audio.transcript) 
   
   print("Add a history message referencing the first turn's audio by ID:")
   print(completion.choices[0].message.audio.id)
   
   # Add a history message referencing the first turn's audio by ID 
   messages.append({ 
       "role": "assistant", 
       "audio": { "id": completion.choices[0].message.audio.id } 
   }) 
   
   # Add the next turn's user message 
   messages.append({ 
       "role": "user", 
       "content": "Very briefly, summarize the favorability." 
   }) 
   
   # Send the follow-up request with the accumulated messages
   completion = client.chat.completions.create( 
       model="gpt-4o-mini-audio-preview", 
       messages=messages
   ) 
   
   print("Very briefly, summarize the favorability.")
   print(completion.choices[0].message.content)

2. Run the Python file.

   python multi-turn.py

API key

1. Create the multi-turn.py file with the following code:

   import base64 
   import os 
   from openai import AzureOpenAI 
   
   # Set environment variables or edit the corresponding values here.
   endpoint = os.environ['AZURE_OPENAI_ENDPOINT']
   api_key = os.environ['AZURE_OPENAI_API_KEY']
   
   client = AzureOpenAI(
       api_version="2025-01-01-preview",  
       api_key=api_key, 
       azure_endpoint=endpoint
   )
   
   # Read and encode audio file  
   with open('dog.wav', 'rb') as wav_reader: 
       encoded_string = base64.b64encode(wav_reader.read()).decode('utf-8') 
   
   # Initialize messages with the first turn's user input 
   messages = [
       { 
           "role": "user", 
           "content": [ 
               { "type": "text", "text": "Describe in detail the spoken audio input." }, 
               { "type": "input_audio", 
                   "input_audio": { 
                       "data": encoded_string, 
                       "format": "wav" 
                   } 
               } 
           ] 
       }] 
   
   # Get the first turn's response 
   
   completion = client.chat.completions.create( 
       model="gpt-4o-mini-audio-preview", 
       modalities=["text", "audio"], 
       audio={"voice": "alloy", "format": "wav"}, 
       messages=messages
   ) 
   
   print("Get the first turn's response:")
   print(completion.choices[0].message.audio.transcript) 
   
   print("Add a history message referencing the first turn's audio by ID:")
   print(completion.choices[0].message.audio.id)
   
   # Add a history message referencing the first turn's audio by ID 
   messages.append({ 
       "role": "assistant", 
       "audio": { "id": completion.choices[0].message.audio.id } 
   }) 
   
   # Add the next turn's user message 
   messages.append({ 
       "role": "user", 
       "content": "Very briefly, summarize the favorability." 
   }) 
   
   # Send the follow-up request with the accumulated messages 
   completion = client.chat.completions.create( 
       model="gpt-4o-mini-audio-preview", 
       messages=messages
   ) 
   
   print("Very briefly, summarize the favorability.")
   print(completion.choices[0].message.content)

2. Run the Python file.

   python multi-turn.py

---

Wait a few moments to get the response.

Output for multi-turn chat completions

The script generates a transcript of the summary of the spoken audio input. Then, it makes a multi-turn chat completion to briefly summarize the spoken audio input.