Deploy a ChatGPT service with Azure OpenAI Service in 6 minutes with PowerShell

OpenAI’s ChatGPT has been one of the most talked about services since its launch on November 30^th, 2022 amongst my professional contacts as well as personal friends. What this Chat Generative Pre-trained Transformer can perform is truly remarkable and opens up so many possibilities in the future. Many of my colleagues have asked me whether I’ve tested it and why I haven’t written any blog posts since Azure released the OpenAI service preview in March 2023. The short answer is that I have performed some testing with it over the last few months but haven’t been able to commit the amount of time I want due to my busy work schedule. I finally had a bit of a breather over the past few weeks so I’ve managed to really try out the following:

1. Pairing with Cognitive Search with a RAG (Retrieval Augmentation Generation) architecture to augment the ChatGPT LLM (Large Language Model) to add data in a Azure Storage Account
2. Deploying front-end UI solutions for the OpenAI service
3. Diving deep into how to secure Azure OpenAI, Cognitive Searches, and data sources with private endpoints and shared private access

It’s amazing how much material there is for #1 and #2 but not as much as I’d like for #3. There is so much Azure’s AI Services can do and I look forward to the projects to come in the following years.

The purpose of this blog post is to show just how fast and easy it is to deploy an Azure OpenAI service with a front-end UI for a private ChatGPT service where internal employees of organizations can safely enter questions with sensitive data. Microsoft is very clear on the usage of the inputs entered in the prompt (https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy):

Your prompts (inputs) and completions (outputs), your embeddings, and your training data:

• are NOT available to other customers.

• are NOT available to OpenAI.
• are NOT used to improve OpenAI models.
• are NOT used to improve any Microsoft or 3rd party products or services.
• are NOT used for automatically improving Azure OpenAI models for your use in your resource (The models are stateless, unless you explicitly fine-tune models with your training data).
• Your fine-tuned Azure OpenAI models are available exclusively for your use.

The Azure OpenAI Service is fully controlled by Microsoft; Microsoft hosts the OpenAI models in Microsoft’s Azure environment and the Service does NOT interact with any services operated by OpenAI (e.g. ChatGPT, or the OpenAI API).

This will put many organizations at ease as I’ve been to one too many dinner parties where I’ve heard people talk about entering data into OpenAI’s ChatGPT to write a letter to HR. I don’t even want to ask what they were entering in there and what else it has been used for.

In any case, I took some time to put together a PowerShell script that prompts for a few questions about what to name the resource group containing all the resources to be created, the name of the Azure OpenAI instance, the LLM model to use, what Azure subscription to use, and it takes care of the rest (Container App, Log Analytics Workspace, etc). I timed the duration of the script and it took 5 minutes and 32 seconds to run. Yes, I understand this is an imperative run rather than declarative. I’m a huge supporter of Infrastructure of Code but I needed something that would allow me to run in any Azure environment to quickly build a demo with all components in a Resource Group so I can easily tear it down by simply deleting the RG.

The deployment is very basic with no private endpoints as I will reserve that for a future post. Here is the simple topology:

With that, let’s get into it now.

Prerequisites

As of October 22, 2023, you may see the Azure OpenAI service as an option in the Azure AI Services blade but attempting to create the service will display the following message:

Azure OpenAI Service is currently available to customers via an application form. The selected subscription has not been enabled for use of the service and does not have quota for any pricing tiers. Click here to request access to Azure OpenAI service.

Clicking on the link will bring you to a Microsoft Form with questions about who you are, why you want to use the service, and what features you would want to turn on:

**I’ve blocked out the content in the screenshot of the form as I am unsure if posting the verbiage is in violation of Microsoft’s policy.

You’ll need to fill out the form, submit it, and receive an approval that is indicated to take up to 10 business days. My form submission took only a day but I assume this can vary so if you fill out the form intend on using the service so you don’t have to wait when you actually want to deploy.

Using a PowerShell script to deploy all the services in 6 minutes (or less)

The PowerShell script I put together can be retrieved from my GitHub repository here: https://github.com/terenceluk/Azure/blob/main/AI%20Services/Deploy-Azure-OpenAI-with-Chatbot-UI.ps1

The script is meant to be executed from the console and it will ask for the user to input:

1. Select a subscription found in the tenant
2. Provide a name for a new Resource Group
3. Provide a name for the OpenAI instance
4. Select a model from the options

The rest of the components such as Container App and Log Analytics will be automatically named (derived from the instance name) and deployed through the remaining script. At the end of a successful run, the browser will automatically launch and the following screen will be displayed:

Azure Resources Deployed

All of the resources for the solution are meant to be deployed into a single resource group for ease of cleanup if it is used for a demo:

The following are screenshots of the resources:

Note that the script will not place the value of the Azure OpenAI key into the environment as a variable, rather, it will store it as a secret that the environment variable references:

I did not create a custom health probe so the one created is the default:

Securing the ChatGPT UI portal with authentication

One of the components I’m still working on is to use BICEP to configure the Container App with Microsoft as an identity provider so the portal would prompt the user for credentials and they are required to log in with a valid account in the tenant’s Entra ID / Azure AD before getting into the portal. If you’d like to turn this on after the script deploys the services, simply navigate to the Container App’s Authentication blade, click on Add identity provider:

Select Microsoft as the Identity provider:

You can leave the settings as default and proceed to create the identity provider:

This will create an App Registration in the tenant’s Entra ID / Azure AD for the Container App to authenticate the user:

Note that you would need to consent the Container App’s App Registration in the portal.azure.com or perform it upon first logging in:

Credits

I want to give a huge thanks to Mckay Wrigley (https://github.com/mckaywrigley) for developing and sharing out his chatbot-ui docker container (https://github.com/mckaywrigley/chatbot-ui) for the world to use. If you search the internet for deployment demonstrations, you are bound to see 9 of the 10 demos using his Chatbot UI. I spent quite a bit of time using Postman to interact with the Azure OpenAI service APIs and as I am not a developer, it would take me quite a bit of time to develop something half as great as Mckay’s.

Final Remarks

One of the behaviors I noticed during the creation and deletion of the services is that when an Azure Open AI instance is deleted, it is dropped into a recycling bin like location and if you decide to deploy another instance in the same name then it will fail. If you have deleted and instance and want to use the same name then use the Manage deleted resources in the Azure OpenAI blade to locate and purge the instance. From what I can tell, the purge is instant and you can proceed to redeploy a new instance with the same name.

I hope this provides anyone out there who is looking to test this great service offering out but haven’t had the time to get started. There are many other great posts I’d like to write about Cognitive Search and the “under the hood view” of the traffic flow but I will save that for another day. Happy chatting!

Generating unique IP visits rendered into a column chart with kusto query for Azure Storage Account hosted website published with an App Gateway

I recently worked with a client who needed to quickly host a static website requiring zero dynamic content and little to no updates for years. Given the short runway available and the team being cost conscious, we opted to use the Static website feature of an Azure Storage Account to publish the website. Other than having to deal with the [I think] widely known WebContentNotFound issue when reloading pages, the service provided an adequate way of hosting the website. There was already an App Gateway in the environment so it was used to provide custom domain and WAF protection capabilities.

A few weeks into the launch of the website, I was asked to generate some statistics for the websites visit and given that I had the Diagnostics settings for the App Gateway set up to send allLogs to a Log Analytics Workspace and the logs captured on the Storage Account wouldn’t provide the real public IP addresses of the inbound traffic, I decided to use KQL to obtain the report requested.

The following are two reports I generated and thought I’d share it in case anyone may be looking for this.

Review visits over a range of days with hours as scale
This report groups unique IP addresses into bins within an hour over the start and end date specified.

Review visits over a range of days
This report groups unique IP addresses for each day over the start and end date specified.

The queries can be retrieved from my GitHub repo: https://github.com/terenceluk/Azure/blob/main/Kusto%20KQL/Azure-App-Gateway-Website-Stats.kusto

Hope this helps anyone who needs this data. The query can easily be changed for any backend service hosting the website and modified for different results.

Using PowerShell to create multiple Azure Storage Account Containers with Metadata using a list on an Excel spreadsheet

I recently worked on a project where we had to create hundreds of containers in multiple Azure Storage Accounts because we needed to the storage account SFTP service and in order to jail users into their own directories, each local SFTP user account needed to have their home folders set to their own containers. This may change in the future but working with this requirement meant many containers had to be created. In addition to creating containers, I also wanted each to have metadata added for the organization that the container belonged to so to reduce the repetitive manual labour, I decided to write a script.

The script can be I created can be found at my following GitHub repo: https://github.com/terenceluk/Azure/blob/main/PowerShell/Create-Storage-Account-Container.ps1

The format of the spreadsheet should look as such:

To handle scenarios where new storage account containers are added at a later time after the script has been executed once, the code will check and skip the creation of the container if it already exists.

Scenarios where I’ve noticed this script will fail is when there are non ASCII characters in the metadata value. These characters include languages such as French (é É) or Microsoft Word dash/hyphen character. I don’t think there is a way to have the PowerShell cmdlet accept these characters.

Below is an example of when these special non ASCII characters are encountered and the metadata value add fails. Note that the container does get created.

Container 17689 has been created.

MethodInvocationException:

Line |

40 | $container.BlobContainerClient.SetMetadata($metadata, $null) …

| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

| Exception calling "SetMetadata" with "2" argument(s): "Retry failed after 6 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.) (Request headers must contain only ASCII characters.)"

Container 17689 has been created.

Hope this helps anyone who might be looking for a script like this.