Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support guidance for load balancing with ACA #360

Open
jmatthiesen opened this issue Aug 28, 2024 · 0 comments
Open

Support guidance for load balancing with ACA #360

jmatthiesen opened this issue Aug 28, 2024 · 0 comments
Assignees

Comments

@jmatthiesen
Copy link
Collaborator

This sample should support guidance for load balancing API requests using ACA, as described in the load balancing articles here:

https://learn.microsoft.com/en-us/dotnet/ai/get-started-app-chat-scaling-with-azure-container-apps?toc=%2Fazure%2Fdeveloper%2Fai%2Ftoc.json&bc=%2Fazure%2Fdeveloper%2Fai%2Fbreadcrumb%2Ftoc.json&tabs=github-codespaces%2Cinitial-deployment)

To support this work, we need to support both the OPENAI_HOST and AZURE_OPENAI_CUSTOM_URL environment variables in the application.

Environment Variable Description
OPENAI_HOST Setting to azure_custom tells the Chat app's backend to use a custom URL for the OpenAI requests.
AZURE_OPENAI_CUSTOM_URL Custom URL to use for OpenAI requests (When OPENAI_HOST variable is set to azure_custom)

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)
@jmatthiesen jmatthiesen added this to the 2024-September milestone Aug 28, 2024
@jmatthiesen jmatthiesen self-assigned this Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant