Configuration
You will need to configure the LLM connection and authorization secrets. You can use remote (like OpenAI, Azure, etc.) and local models (like Ollama, Jan, LMStudio, etc.) with GenAIScript.
Model selection
The model used by the script is configured through the model
field in the script
function.
The model name is formatted as provider:model-name
, where provider
is the LLM provider
and the model-name
is provider specific.
script({ model: "openai:gpt-4o",})
Large, small, vision models
You can also use the small
, large
, vision
model aliases to use the default configured small, large and vision-enabled models.
Large models are typically in the OpenAI gpt-4 reasoning range and can be used for more complex tasks.
Small models are in the OpenAI gpt-4o-mini range, and are useful for quick and simple tasks.
script({ model: "small" })
script({ model: "large" })
The model aliases can also be overridden from the cli run command, or environment variables or configuration file. Learn more about model aliases.
genaiscript run ... --model largemodelid --small-model smallmodelid
or by adding the GENAISCRIPT_MODEL_LARGE
and GENAISCRIPT_MODEL_SMALL
environment variables.
GENAISCRIPT_MODEL_LARGE="azure_serverless:..."GENAISCRIPT_MODEL_SMALL="azure_serverless:..."GENAISCRIPT_MODEL_VISION="azure_serverless:..."
You can also configure the default aliases for a given LLM provider by using the provider
argument.
The default are documented in this page and printed to the console output.
script({ provider: "openai" })
genaiscript run ... --provider openai
Model aliases
In fact, you can define any alias for your model (only alphanumeric characters are allowed)
through environment variables of the name GENAISCRIPT_MODEL_ALIAS
where ALIAS
is the alias you want to use.
GENAISCRIPT_MODEL_TINY=...
Model aliases are always lowercased when used in the script.
script({ model: "tiny" })
.env
file
GenAIScript uses a .env
file to load secrets and configuration information into the process environment variables.
Create or update a
.gitignore
file in the root of your project and make it sure it includes.env
. This ensures that you do not accidentally commit your secrets to your source control..gitignore ....envCreate a
.env
file in the root of your project.- .gitignore
- .env
Update the
.env
file with the configuration information (see below).
Custom .env file location
You can specify a custom .env
file location through the CLI or an environment variable.
- by adding the
--env <file>
argument to the CLI.
npx genaiscript ... --env .env.local
- by setting the
GENAISCRIPT_ENV_FILE
environment variable.
GENAISCRIPT_ENV_FILE=".env.local" npx genaiscript ...
- by specifying the
.env
file location in a configuration file.
envFile: ~/.env.genaiscript
No .env file
If you do not want to use a .env
file, make sure to populate the environment variables
of the genaiscript process with the configuration values.
Here are some common examples:
- Using bash syntax
OPENAI_API_KEY="value" npx --yes genaiscript run ...
- GitHub Action configuration
run: npx --yes genaiscript run ...env: OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
configure
command
The configure command is an interactive command to configure and validate the LLM connections.
npx genaiscript configure
OpenAI
openai
is the OpenAI chat model provider.
It uses the OPENAI_API_...
environment variables.
Upgrade your account to get access to the models. You will get 404s if you do not have a paying account.
Create a new secret key from the OpenAI API Keys portal.
Update the
.env
file with the secret key..env OPENAI_API_KEY=sk_...Find the model you want to use from the OpenAI API Reference or the OpenAI Chat Playground.
Set the
model
field inscript
to the model you want to use.script({model: "openai:gpt-4o",...})
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | gpt-4o |
small | gpt-4o-mini |
vision | gpt-4o |
vision_small | gpt-4o-mini |
embeddings | text-embedding-3-small |
reasoning | o1 |
reasoning_small | o1-mini |
transcription | whisper-1 |
speech | tts-1 |
GitHub Models
The GitHub Models provider, github
, allows running models through the GitHub Marketplace.
This provider is useful for prototyping and subject to rate limits
depending on your subscription.
script({ model: "github:gpt-4o" })
Create a GitHub personal access token. The token should not have any scopes or permissions.
Update the
.env
file with the token..env GITHUB_TOKEN=...
To configure a specific model,
Open the GitHub Marketplace and find the model you want to use.
Copy the model name from the Javascript/Python samples
const modelName = "Phi-3-mini-4k-instruct"to configure your script.
script({model: "github:Phi-3-mini-4k-instruct",})
If you are already using GITHUB_TOKEN
variable in your script and need a different one
for GitHub Models, you can use the GITHUB_MODELS_TOKEN
variable instead.
o1-preview
and o1-mini
models
Currently these models do not support streaming and system prompts. GenAIScript handles this internally.
script({ model: "github:o1-mini",})
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | gpt-4o |
small | gpt-4o-mini |
vision | gpt-4o |
embeddings | text-embedding-3-small |
reasoning | o1-preview |
reasoning_small | o1-mini |
Limitations
- Smaller context windows, and rate limiting
- listModels
- logprobs (and top logprobs) ignored
- Ignore prediction of output tokens
- topLogprobs
Azure OpenAI
The Azure OpenAI provider, azure
uses the AZURE_OPENAI_...
environment variables.
You can use a managed identity (recommended) or an API key to authenticate with the Azure OpenAI service.
You can also use a service principal as documented in automation.
script({ model: "azure:deployment-id" })
Managed Identity (Entra ID)
Open your Azure OpenAI resource in the Azure Portal
Navigate to Access Control (IAM), then View My Access. Make sure your user or service principal has the Cognitive Services OpenAI User/Contributor role. If you get a
401
error, click on Add, Add role assignment and add the Cognitive Services OpenAI User role to your user.Navigate to Resource Management, then Keys and Endpoint.
Update the
.env
file with the endpoint..env AZURE_OPENAI_API_ENDPOINT=https://....openai.azure.comNavigate to deployments and make sure that you have your LLM deployed and copy the
deployment-id
, you will need it in the script.Open a terminal and login with Azure CLI.
Terminal window az loginUpdate the
model
field in thescript
function to match the model deployment name in your Azure resource.script({model: "azure:deployment-id",...})
Listing models
In order to allow GenAIScript to list deployments in your Azure OpenAI service, you need to provide the Subscription ID and you need to use Microsoft Entra!.
Open the Azure OpenAI resource in the Azure Portal, open the Overview tab and copy the Subscription ID.
Update the
.env
file with the subscription id..env AZURE_OPENAI_SUBSCRIPTION_ID="..."Test your configuration by running
Terminal window npx genaiscript models azure
Custom credentials
In some situations, the default credentials chain lookup may not work.
In that case, you can specify an additional environment variable AZURE_OPENAI_API_CREDENTIALS
with the type of credential that should be used.
AZURE_OPENAI_API_CREDENTIALS=cli
The types are mapped directly to their @azure/identity credential types:
cli
-AzureCliCredential
env
-EnvironmentCredential
powershell
-AzurePowerShellCredential
devcli
-AzureDeveloperCliCredential
managedidentity
-ManagedIdentityCredential
Custom token scopes
The default token scope for Azure OpenAI access is https://cognitiveservices.azure.com/.default
.
You can override this value using the AZURE_OPENAI_TOKEN_SCOPES
environment variable.
AZURE_OPENAI_TOKEN_SCOPES=...
API Version
GenAIScript maintains a default API version to access Azure OpenAI. You can override this value using the AZURE_OPENAI_API_VERSION
environment variable.
AZURE_OPENAI_API_VERSION=2025-01-01-preview
You can also override the API version on a per-deployment basis by settings the AZURE_OPENAI_API_VERSION_<deployment-id>
environment variable (where deployment-id is capitalized).
AZURE_OPENAI_API_VERSION_GPT-4O=2025-01-01-preview
API Key
Open your Azure OpenAI resource and navigate to Resource Management, then Keys and Endpoint.
Update the
.env
file with the secret key (Key 1 or Key 2) and the endpoint..env AZURE_OPENAI_API_KEY=...AZURE_OPENAI_API_ENDPOINT=https://....openai.azure.comThe rest of the steps are the same: Find the deployment name and use it in your script,
model: "azure:deployment-id"
.
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|
Limitations
- listModels
- Ignore prediction of output tokens
Azure AI Serverless Deployments
You can deploy “serverless” models through Azure AI Foundry and pay as you go per token. You can browse the Azure AI model catalog and use the serverless API filter to see the available models.
There are two types of serverless deployments that require different configurations: OpenAI models and all other models.
The OpenAI models, like gpt-4o
, are deployed to .openai.azure.com
endpoints,
while the Azure AI models, like Meta-Llama-3.1-405B-Instruct
are deployed to .models.ai.azure.com
endpoints.
They are configured slightly differently.
Azure AI OpenAI Serverless
The azure_serverless
provider supports OpenAI models deployed through the Azure AI Foundry serverless deployments.
It supports both Entra ID and key-based authentication.
script({ model: "azure_serverless:deployment-id" })
Managed Identity (Entra ID)
Open https://ai.azure.com/ and open the Deployments page.
Deploy a base model from the catalog. You can use the
Deployment Options
->Serverless API
option to deploy a model as a serverless API.Deploy an OpenAI base model. This will also create a new Azure OpenAI resource in your subscription (which may be invisible to you, more later).
Update the
.env
file with the deployment endpoint in theAZURE_SERVERLESS_OPENAI_API_ENDPOINT
variable..env AZURE_SERVERLESS_OPENAI_API_ENDPOINT=https://....openai.azure.comGo back to the Overview tab in your Azure AI Foundry project and click on Open in Management center.
Click on the Azure OpenAI Service resource, then click on the Resource external link which will take you back to the (underlying) Azure OpenAI service in Azure Portal.
Navigate to Access Control (IAM), then View My Access. Make sure your user or service principal has the Cognitive Services OpenAI User/Contributor role. If you get a
401
error, click on Add, Add role assignment and add the Cognitive Services OpenAI User role to your user.
At this point, you are ready to login with the Azure CLI and use the managed identity.
Install the Azure CLI.
Open a terminal and login
Terminal window az login
API Key
Open your Azure OpenAI resource and navigate to Resource Management, then Keys and Endpoint.
Update the
.env
file with the endpoint and the secret key (Key 1 or Key 2) and the endpoint..env AZURE_SERVERLESS_OPENAI_API_ENDPOINT=https://....openai.azure.comAZURE_SERVERLESS_OPENAI_API_KEY=...
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | gpt-4o |
small | gpt-4o-mini |
vision | gpt-4o |
vision_small | gpt-4o-mini |
reasoning | o1 |
reasoning_small | o1-mini |
Limitations
- listModels
- Ignore prediction of output tokens
Azure AI Models
The azure_serverless_models
provider supports non-OpenAI models deployed through the Azure AI Foundary serverless deployments.
script({ model: "azure_serverless_models:deployment-id" })
Managed Identity (Entra ID)
Open your Azure AI Project resource in the Azure Portal
Navigate to Access Control (IAM), then View My Access. Make sure your user or service principal has the Azure AI Developer role. If you get a
401
error, click on Add, Add role assignment and add the Azure AI Developer role to your user.Configure the Endpoint Target URL as the
AZURE_SERVERLESS_MODELS_API_ENDPOINT
..env AZURE_SERVERLESS_MODELS_API_ENDPOINT=https://...models.ai.azure.comNavigate to deployments and make sure that you have your LLM deployed and copy the Deployment Info name, you will need it in the script.
Update the
model
field in thescript
function to match the model deployment name in your Azure resource.script({model: "azure_serverless:deployment-info-name",...})
API Key
Open https://ai.azure.com/ and open the Deployments page.
Deploy a base model from the catalog. You can use the
Deployment Options
->Serverless API
option to deploy a model as a serverless API.Configure the Endpoint Target URL as the
AZURE_SERVERLESS_MODELS_API_ENDPOINT
variable and the key inAZURE_SERVERLESS_MODELS_API_KEY
in the.env
file**.**.env AZURE_SERVERLESS_MODELS_API_ENDPOINT=https://...models.ai.azure.comAZURE_SERVERLESS_MODELS_API_KEY=...Find the deployment name and use it in your script,
model: "azure_serverless_models:deployment-id"
.
Support for multiple inference deployements
You can update the AZURE_SERVERLESS_MODELS_API_KEY
with a list of deploymentid=key
pairs to support multiple deployments (each deployment has a different key).
AZURE_SERVERLESS_MODELS_API_KEY="model1=key1model2=key2model3=key3"
Limitations
- listModels
- Ignore prediction of output tokens
Google AI
The google
provider allows you to use Google AI models. It gives you access
Open Google AI Studio and create a new API key.
Update the
.env
file with the API key..env GEMINI_API_KEY=...Find the model identifier in the Gemini documentation and use it in your script or cli with the
google
provider....const model = genAI.getGenerativeModel({model: "gemini-1.5-pro-latest",});...then use the model identifier in your script.
script({ model: "google:gemini-1.5-pro-latest" })
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | gemini-1.5-flash-latest |
small | gemini-1.5-flash-latest |
vision | gemini-1.5-flash-latest |
long | gemini-1.5-flash-latest |
reasoning | gemini-2.0-flash-thinking-exp-1219 |
reasoning_small | gemini-2.0-flash-thinking-exp-1219 |
embeddings | text-embedding-004 |
Limitations
- Uses OpenAI compatibility layer
- listModels
- logprobs (and top logprobs) ignored
- Ignore prediction of output tokens
- Seed ignored
- Tools implemented as fallback tools automatically.
- topLogprobs
GitHub Copilot Chat Models
If you have access to GitHub Copilot Chat in Visual Studio Code, GenAIScript will be able to leverage those language models as well.
This mode is useful to run your scripts without having a separate LLM provider or local LLMs. However, those models are not available from the command line and have additional limitations and rate limiting defined by the GitHub Copilot platform.
There is no configuration needed as long as you have GitHub Copilot installed and configured in Visual Studio Code.
You can force using this model by using github_copilot_chat:*
as a model name.
Install GitHub Copilot Chat (emphasis on Chat)
- run your script
Confirm that you are allowing GenAIScript to use the GitHub Copilot Chat models.
select the best chat model that matches the one you have in your script
(This step is skipped if you already have mappings in your settings)
The mapping of GenAIScript model names to Visual Studio Models is stored in the settings.
Anthropic
The anthropic
provider access Anthropic models. Anthropic is an AI research company that offers powerful language models, including the Claude series.
script({ model: "anthropic:claude-2.1" })
To use Anthropic models with GenAIScript, follow these steps:
Sign up for an Anthropic account and obtain an API key from their console.
Add your Anthropic API key to the
.env
file:.env ANTHROPIC_API_KEY=sk-ant-api...Find the model that best suits your needs by visiting the Anthropic model documentation.
Update your script to use the
model
you choose.script({...model: "anthropic:claude-3-5-sonnet-20240620",})
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | claude-3-5-sonnet-latest |
small | claude-3-5-haiku-latest |
vision | claude-3-5-sonnet-latest |
Limitations
- listModels
- logprobs (and top logprobs) ignored
- Ignore prediction of output tokens
- topLogprobs
Anthropic Bedrock
The anthropic_bedrock
provider accesses Anthropic models on Amazon Bedrock. You can find the model names in the Anthropic model documentation.
GenAIScript assumes that you have configured AWS credentials in a way that the AWS Node SDK will recognise.
script({ model: "anthropic_bedrock:anthropic.claude-3-sonnet-20240229-v1:0" })
Hugging Face
The huggingface
provider allows you to use Hugging Face Models using Text Generation Inference.
script({ model: "huggingface:microsoft/Phi-3-mini-4k-instruct" })
To use Hugging Face models with GenAIScript, follow these steps:
Sign up for a Hugging Face account and obtain an API key from their console. If you are creating a Fined Grained token, enable the Make calls to the serverless inference API option.
Add your Hugging Face API key to the
.env
file asHUGGINGFACE_API_KEY
,HF_TOKEN
orHUGGINGFACE_TOKEN
variables..env HUGGINGFACE_API_KEY=hf_...Find the model that best suits your needs by visiting the HuggingFace models.
Update your script to use the
model
you choose.script({...model: "huggingface:microsoft/Phi-3-mini-4k-instruct",})
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | meta-llama/Llama-3.3-70B-Instruct |
small | microsoft/phi-4 |
vision | meta-llama/Llama-3.2-11B-Vision-Instruct |
embeddings | nomic-ai/nomic-embed-text-v1.5 |
Limitations
- Uses OpenAI compatibility layer
- listModels
- Ignore prediction of output tokens
Mistral AI
The mistral
provider allows you to use Mistral AI Models
using the Mistral API.
script({ model: "mistral:mistral-large-latest" })
Sign up for a Mistral AI account and obtain an API key from their console.
Add your Mistral AI API key to the
.env
file:.env MISTRAL_API_KEY=...Update your script to use the
model
you choose.script({...model: "mistral:mistral-large-latest",})
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | mistral-large-latest |
small | mistral-small-latest |
vision | pixtral-large-latest |
Limitations
- Ignore prediction of output tokens
Alibaba Cloud
The alibaba
provider access the Alibaba Cloud models.
script({ model: "alibaba:qwen-max",})
Sign up for a Alibaba Cloud account and obtain an API key from their console.
Add your Alibaba API key to the
.env
file:.env ALIBABA_API_KEY=sk_...Find the model that best suits your needs by visiting the Alibaba models.
Update your script to use the
model
you choose.script({...model: "alibaba:qwen-max",})
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | qwen-max |
small | qwen-turbo |
long | qwen-plus |
embeddings | text-embedding-v3 |
Limitations
- Uses OpenAI compatibility layer
- listModels
- Ignore prediction of output tokens
- Tools implemented as fallback tools automatically.
Ollama
Ollama is a desktop application that lets you download and run models locally.
Running tools locally may require additional GPU resources depending on the model you are using.
Use the ollama
provider to access Ollama models.
Start the Ollama application or
Terminal window ollama serveUpdate your script to use the
ollama:phi3.5
model (or any other model or from Hugging Face).script({...,model: "ollama:phi3.5",})GenAIScript will automatically pull the model, which may take some time depending on the model size. The model is cached locally by Ollama.
If Ollama runs on a server or a different computer or on a different port, you have to configure the
OLLAMA_HOST
environment variable to connect to a remote Ollama server..env OLLAMA_HOST=https://<IP or domain>:<port>/ # server urlOLLAMA_HOST=0.0.0.0:12345 # different port
You can specify the model size by adding the size to the model name, like ollama:llama3.2:3b
.
script({ ..., model: "ollama:llama3.2:3b",})
Ollama with Hugging Face models
You can also use GGUF models from Hugging Face.
script({ ..., model: "ollama:hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF",})
Ollama with Docker
You can conviniately run Ollama in a Docker container.
- start the Ollama container
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
- stop and remove the Ollama containers
docker stop ollama && docker rm ollama
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
embeddings | nomic-embed-text |
Limitations
- Uses OpenAI compatibility layer
- logit_bias ignored
- Ignore prediction of output tokens
DeepSeek
deepseek
is the DeepSeek chat model provider.
It uses the DEEPSEEK_API_...
environment variables.
Create a new secret key from the DeepSeek API Keys portal.
Update the
.env
file with the secret key..env DEEPSEEK_API_KEY=sk_...Set the
model
field inscript
todeepseek:deepseek:deepseek-chat
which is currently the only supported model.script({model: "deepseek:deepseek-chat",...})
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
large | deepseek-chat |
small | deepseek-chat |
vision | deepseek-chat |
LM Studio
The lmstudio
provider connects to the LMStudio headless server.
and allows to run local LLMs.
Install LMStudio (v0.3.5+)
Open LMStudio
Open the Model Catalog, select your model and load it at least once so it is downloaded locally.
Open the settings (Gearwheel icon) and enable Enable Local LLM Service.
GenAIScript assumes the local server is at
http://localhost:1234/v1
by default. Add aLMSTUDIO_API_BASE
environment variable to change the server URL..env LMSTUDIO_API_BASE=http://localhost:2345/v1
Find the model API identifier in the dialog of loaded models then use that identifier in your script:
script({ model: "lmstudio:llama-3.2-1b-instruct",})
- GenAIScript uses the LMStudio CLI to pull models on demand.
- Specifiying the quantization is currently not supported.
Aliases
The following model aliases are attempted by default in GenAIScript.
Alias | Model identifier |
---|---|
embeddings | text-embedding-nomic-embed-text-v1.5 |
Limitations
- Ignore prediction of output tokens
LM Studio and Hugging Face Models
Follow this guide to load Hugging Face models into LMStudio.
Jan
The jan
provider connects to the Jan local server.
Open Jan and download the models you plan to use. You will find the model identifier in the model description page.
Click on the Local API Server icon (lower left), then Start Server.
Keep the desktop application running!
To use Jan models, use the jan:modelid
syntax.
If you change the default server URL, you can set the JAN_API_BASE
environment variable.
JAN_API_BASE=http://localhost:1234/v1
Limitations
- Ignore prediction of output tokens
- top_p ignored
LocalAI
LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It uses free Open Source models and it runs on CPUs.
LocalAI acts as an OpenAI replacement, you can see the model name mapping
used in the container, like gpt-4
is mapped to phi-2
.
Install Docker. See the LocalAI documentation for more information.
Update the
.env
file and set the api type tolocalai
..env OPENAI_API_TYPE=localai
To start LocalAI in docker, run the following command:
docker run -p 8080:8080 --name local-ai -ti localai/localai:latest-aio-cpudocker start local-aidocker statsecho "LocalAI is running at http://127.0.0.1:8080"
Llamafile
https://llamafile.ai/ is a single file desktop application that allows you to run an LLM locally.
The provider is llamafile
and the model name is ignored.
LLaMA.cpp
LLaMA.cpp also allow running models locally or interfacing with other LLM vendors.
Update the
.env
file with the local server information..env OPENAI_API_BASE=http://localhost:...
OpenRouter
You can configure the OpenAI provider to use the OpenRouter service instead
by setting the OPENAI_API_BASE
to https://openrouter.ai/api/v1
.
You will also need an api key.
OPENAI_API_BASE=https://openrouter.ai/api/v1OPENAI_API_KEY=...
Then use the OpenRouter model name in your script:
script({ model: "openai:openai/gpt-4o-mini" })
By default, GenAIScript will set the site URL and name to GenAIScript
but you can override these settings with your own values:
OPENROUTER_SITE_URL=... # populates HTTP-Referer headerOPENROUTER_SITE_NAME=... # populate X-Title header
LiteLLM
The LiteLLM proxy gateway provides a OpenAI compatible API for running models locally.
Configure the LITELLM_...
keys to set the key and optionally the base url.
LITELLM_API_KEY="..."#LITELLM_API_BASE="..."
Hugging Face Transformer.js
This transformers
provider runs models on device using Hugging Face Transformers.js.
The model syntax is transformers:<repo>:<dtype>
where
repo
is the model repository on Hugging Face,dtype
is the quantization type.
script({ model: "transformers:onnx-community/Qwen2.5-Coder-0.5B-Instruct:q4",})
The default transformers device is cpu
,
but you can changing it using HUGGINGFACE_TRANSFORMERS_DEVICE
environment variable.
HUGGINGFACE_TRANSFORMERS_DEVICE=gpu
Limitations
- Ignore prediction of output tokens
Whisper ASR WebServices
This whisperasr
provider allows to configure a transcription
task to use the Whisper ASR WebService project.
const transcript = await transcribe("video.mp4", { model: "whisperasr:default",})
This whisper service can run locally or in a docker container (see documentation).
docker run -d -p 9000:9000 -e ASR_MODEL=base -e ASR_ENGINE=openai_whisper onerahmet/openai-whisper-asr-webservice:latest
You can also override the transcription
model alias to change the default model used by transcribe
.
Model specific environment variables
You can provide different environment variables
for each named model by using the PROVIDER_MODEL_API_...
prefix or PROVIDER_API_...
prefix.
The model name is capitalized and
all non-alphanumeric characters are converted to _
.
This allows to have various sources of LLM computations
for different models. For example, to enable the ollama:phi3
model running locally, while keeping the default openai
model connection information.
OLLAMA_PHI3_API_BASE=http://localhost:11434/v1
Running behind a proxy
You can set the HTTP_PROXY
and/or HTTPS_PROXY
environment variables to run GenAIScript behind a proxy.
HTTP_PROXY=http://proxy.example.com:8080
Checking your configuration
You can check your configuration by running the genaiscript info env
command.
It will display the current configuration information parsed by GenAIScript.
genaiscript info env
Next steps
Write your first script.