Tutorial: Connect AKS to Azure OpenAI

This tutorial shows you how to connect your Azure Kubernetes Service (AKS) applications to Azure OpenAI using Service Connector with workload identity authentication. You'll establish credential-free connections by deploying a sample Python application that communicates with the Azure OpenAI.

You'll complete the following tasks:

Create an AKS cluster and Azure OpenAI resource with GPT-4 model
Configure Service Connector to establish the connection with workload identity
Clone a sample application
Build and push container images to Azure Container Registry
Deploy the application to AKS and verify the connection
Clean up resources

Prerequisites

An Azure account with an active subscription. Create an account for free.
- Use the Bash environment in Azure Cloud Shell. For more information, see Get started with Azure Cloud Shell.
- If you prefer to run CLI reference commands locally, install the Azure CLI. If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see How to run the Azure CLI in a Docker container.
  - If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see Authenticate to Azure using Azure CLI.
  - When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see Use and manage extensions with the Azure CLI.
  - Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.
Docker and kubectl to manage container images and Kubernetes resources.
A basic understanding of containers and AKS. Get started from preparing an application for AKS.
Access permissions to create Azure OpenAI resources and deploy models.

Create Azure OpenAI and AKS resources

You start this tutorial by creating several Azure resources.

Create a resource group for this tutorial.

az group create \ --name MyResourceGroup \ --location eastus

Create an AKS cluster with the following command, or by referring to the AKS quickstart. In this tutorial, we create the service connection and pod definition and deploy the sample application to this cluster.
```
az aks create \ --resource-group MyResourceGroup \ --name MyAKSCluster \ --enable-managed-identity \ --node-count 1 \ --generate-ssh-keys 
```

Connect to the cluster using the az aks get-credentials command.

az aks get-credentials \ --resource-group MyResourceGroup \ --name MyAKSCluster

Create an Azure OpenAI resource using the az cognitiveservices account create command. Optionally refer to this tutorial for more instructions. Azure OpenAI is the target service that the AKS cluster will connect to.
```
az cognitiveservices account create \ --resource-group MyResourceGroup \ --name MyOpenAI \ --location eastus \ --kind OpenAI \ --sku s0 \ --custom-domain myopenai \ --subscription <SubscriptionID> 
```

Deploy a model with the az cognitiveservices deployment create command. The model is used in the sample application to test the connection.

az cognitiveservices account deployment create \ --resource-group MyResourceGroup \ --name MyOpenAI \ --deployment-name MyModel \ --model-name gpt-4 \ --model-version 0613 \ --model-format OpenAI \ --sku-name "Standard" \ --capacity 1

Create an Azure Container Registry (ACR) to store the containerized sample application. Use the az acr create command, or refer to this tutorial.
```
az acr create \ --resource-group MyResourceGroup \ --name myregistry \ --sku Standard 
```
Enable anonymous pull using az acr update command so that the AKS cluster can consume the images in the registry.
```
az acr update \ --resource-group MyResourceGroup \ --name myregistry \ --anonymous-pull-enabled 
```
Create a user-assigned managed identity with the az identity create command, or by referring to this tutorial. When the connection is created, the user-assigned managed identity is used to enable the workload identity for AKS workloads.
```
az identity create \ --resource-group MyResourceGroup \ --name MyIdentity 
```

Create a service connection from AKS to Azure OpenAI

Create a service connection between an AKS cluster and Azure OpenAI in the Azure portal or the Azure CLI.

Refer to the AKS service connection quickstart for instructions to create a new connection and fill in the settings referring to the examples in the following table. Leave all other settings with their default values.

Basics tab:

Setting	Example value	Description
Kubernetes namespace	default	The Kubernetes namespace.
Service type	OpenAI Service	The target service type.
Connection name	openai_conn	Use the connection name provided by Service Connector or choose your own connection name.
Subscription	My Subscription	The Azure subscription containing your Azure OpenAI resource.
OpenAI	MyOpenAI	The target Azure OpenAI resource you want to connect to.
Client type	Python	The programming language or framework for the connection configuration.

Authentication tab:

Authentication Setting	Example value	Description
Authentication type	Workload Identity	The authentication method to connect the app to Azure OpenAI. Workload identity is recommended for enhanced security. Alternative methods include connection string and service principal, and require credential management considerations.
Subscription	My Subscription	The subscription that contains the user-assigned managed identity.
User assigned managed identity	myidentity	The user-assigned managed identity that enables workload identity authentication for the AKS cluster.

Once the connection is created, you can view its details in the Service Connector pane.

Create a service connection from AKS to the Azure OpenAI resource by running the az aks connection create cognitiveservices command in the Azure CLI.

az aks connection create cognitiveservices \ --workload-identity <user-identity-resource-id>

When using the above command, Service Connector prompts you to specify the AKS resource group, AKS cluster name, target service resource group, cognitive service account name, and user-assigned identity resource ID step by step.

Alternatively, you can provide the complete command directly:

az aks connection create cognitiveservices \ --workload-identity <user-identity-resource-id> \ --resource-group <aks-cluster-resource-group> \ --name <aks-cluster-name> \ --target-resource-group <target-cognitive-services-resource-group> \ --account <target-cognitive-services-account>

Note

This tutorial uses workload identity for enhanced security. For development scenarios, you can also use connection string authentication by omitting the --workload-identity parameter and using --secret instead.

Clone Python sample application

Clone the sample repository:

git clone https://github.com/Azure-Samples/serviceconnector-aks-samples.git

Go to the repository's sample folder for Azure OpenAI:

cd serviceconnector-aks-samples/azure-openai-workload-identity

Replace the <MyModel> placeholder in the app.py file with the model name we deployed.

Build and push container images to Azure Container Registry

Build and push the images to your container registry using the Azure CLI az acr build command.
```
az acr build --registry myregistry --image sc-demo-openai-identity:latest ./ 
```
View the images in your container registry using the az acr repository list command.
```
az acr repository list --name myregistry --output table 
```

Deploy and test AKS to Azure OpenAI connection

Replace the placeholders in the pod.yaml file in the azure-openai-workload-identity folder.
- Replace <YourContainerImage> with the name of the image you built earlier. For example <myregistry>.azurecr.io/<sc-demo-openai-identity>:<latest>.
- Replace <ServiceAccountCreatedByServiceConnector> with the service account name. It can be found in the Azure portal, in the Service Connector pane.
- Replace <SecretCreatedByServiceConnector> with the secret name. It can be found in the Azure portal, in the Service Connector pane.
Deploy the pod to your cluster with the kubectl apply command, which creates a pod named sc-demo-openai-identity in the default namespace of your AKS cluster. Install kubectl locally using the az aks install-cli command if it isn't installed.
```
kubectl apply -f pod.yaml 
```
Check if the deployment was successful by viewing the pod with kubectl.
```
kubectl get pod/sc-demo-openai-identity 
```
Check that connection is established by viewing the logs with kubectl.
```
kubectl logs pod/sc-demo-openai-identity 
```

Clean up resources

If you no longer need the resources created in this tutorial, clean them up by deleting the resource group.

az group delete \ --resource-group MyResourceGroup

Last updated on 2025-09-30

Share via