Azure model provider

Azure OpenAI is a great option if you’re looking to use the OpenAI model but you need more advanced configuration options and enhanced data privacy. With this advanced configuration comes more complexity in setup, so choose wisely.

Information

The Azure OpenAI service can be used in a HIPAA-compliant manner.

To use Azure OpenAI with AI Assistant, you’ll need set up the API key and reference the correct deployments in the AI Assistant service configuration file.

Environment variables

  • AZURE_API_KEY — The API key for the Azure OpenAI service. You can retrieve your keys once you’ve created an instance. See the Azure quickstart guide for more information:

services:
  ai-assistant:
    environment:
      - AZURE_API_KEY=your-azure-api-key
  ...

Refer to the configuration options guide for all the configurable environment variables.

Service configuration file

To specify the Azure OpenAI models used by AI Assistant, you’ll need to create a service configuration file, as explained in the model-provider configuration guide.

Our current chat model suggestion for the Azure OpenAI model is gpt4o-mini, along with an embedding model of text-embedding-3-small.

Here’s an example YAML configuration file using the Azure OpenAI setup:

version: '1'

aiServices:
   chat:
      provider:
         name: 'azure'
         instanceName: 'your-instance-name'
         apiKey: 'your-azure-api-key' # Optional
      model: 'your-model-deployment-name'
   textEmbeddings:
      provider:
         name: 'azure'
         instanceName: 'your-instance-name'
         apiKey: 'your-azure-api-key' # Optional
      model: 'your-embedding-model-deployment-name'

  • provider:

    • name: The name of the provider. Set this to azure.

    • instanceName: The name of your Azure OpenAI instance. Follow the Azure OpenAI instance guide to set up your instance.

    • apiKey: The API key for the Azure OpenAI service. You can retrieve your keys once you’ve created an instance. See the Azure quickstart guide for more information.

  • model: The name of the model deployment you want to use for chat. Follow the Azure OpenAI deployment guide to deploy your model.

Pricing

As you’re providing your own OpenAI API key, you’ll be subject to all the costs related to using the OpenAI or Azure OpenAI service. The following information is provided to give you a rough estimate of how much to expect this to cost.

Document ingestion

The first time you use a document with AI Assistant, it’ll be passed through an ingestion process. This involves converting the textual contents to vector embeddings, creating a summary of the document, and various other classification processes.

You can expect this process to cost ~$0.005 per document, but this price can vary, depending of the amount of content in the document.

That means if you have a document with 10,000 characters, it’ll cost $0.0003. For a large document with 1,000,000 characters, it’ll cost closer to $0.01.

Chat

There are various agents embedded into AI Assistant, which means that estimating can be difficult because it’s highly dependent on user interaction.

Simple Q&A and follow-up requests

A simple question like “What is the best programming language in the world?” will result in a cost of roughly $0.0005. You may see variations of up to $0.001 and as low as $0.0001.

Similarly, expect to see a similar cost when making follow-up requests, such as “Reformat that as an email newsletter.”

Redaction

Redaction is a much more complicated process, meaning the cost is higher per request. We limit the maximum number of pages each redaction request can process to 10 pages, with the optional suggested prompt to continue processing the proceeding 10 pages. We do this to limit wasteful requests.

For a single redaction request such as “Redact all personal information from the document,” you’ll see a cost of roughly $0.008, with variations between $0.01 and $0.005.