AI configuration

By default, AI Assistant operates using models from OpenAI. To connect AI Assistant to OpenAI, you’ll need to pass your OpenAI API key as an environment variable to the AI Assistant service.

If you’re concerned with the security aspects of OpenAI, or if you want to keep your data in a certain region, you may want to configure a different model provider, such as Azure, AWS Bedrock, or a locally served models.

This guide will introduce you to the configuration file used to customize the models and providers during AI Assistant operation.

Mounting the configuration file

Before launching into the default OpenAI configuration, AI Assistant will first check for a service-config.yml file located at the root of the AI Assistant directory. You can mount a file in this location of your service in the Docker Compose file:

ai-assistant:
  volumes:
    - /path-to-service-config-yaml-on-host:/service-config.yml

Service configuration file

The service configuration file is made up of two sections: first, the version of the file, and second the aiServices. Each item in aiServices represents a service that performs a specific function in AI Assistant, and which requires both a provider and a model:

version: '1'

aiServices:
   chat:
      provider:
         name: 'azure'
         instanceName: 'ci-testing'
      model: 'gpt4o-mini'
   textEmbeddings:
      provider:
         name: 'azure'
         instanceName: 'ci-testing'
      model: 'text-embedding-3-small'

Currently, there are two aiServices required.

chat

chat is the AI service used for all large language model (LLM) inference. To achieve reliable and reproducible results, the model this service uses must be a modern, large, and capable LLM such as GPT-4o. For more model selection advice from each provider, navigate to the provider’s guides in the following section.

textEmbeddings

textEmbeddings is the AI service used to convert the content of your document into numerical vectors (mathematical representations). These vectors help AI Assistant quickly find and recall relevant parts of your documents during conversations.

While you can choose from several embedding models, it’s important to note that this choice is permanent for your documents — changing models would require reprocessing all documents in your AI Assistant database.

For guidance on selecting the best embedding model for your needs, refer to the provider guides below.

Providers

For each service in aiServices, you may define one of the following providers:

  • openai (guide) — Choose this for a wide range of performant models at cheap prices.

  • azure (guide) — Use OpenAI models in the Azure environment with more region and data security features.

  • bedrock (guide) — Choose from a range of open and closed models while keeping your data in the AWS ecosystem.

  • openai-compat (guide) — Use any OpenAI API-compatible inference framework, such as vLLM, Hugging Face TGI, or Ollama.

Navigate to each of the guides to discover configuration options, key requirements, and model suggestions.