2024-04-25: Bugfixes for free sku, ADLS2, storageUrl

Today's release includes a variety of bugfixes:

What's Changed

• Fix gunicorn config to use single worker for free sku by @pamelafox in #1551
• Adding missing adls2 variables to azd pipeline by @pamelafox in #1544
• Adding user-upload as a feature in sample comparison by @pamelafox in #1542
• Disable OpenAI key access by @pamelafox in #1555
• Add storageurl field if missing in index by @pamelafox in #1556

Full Changelog: 2024-04-19...2024-04-25

2024-04-19: Improved security for user-uploaded documents

This release improves the security of the optional user upload mechanism by adding a storageUrl field to disambiguate between user-uploaded files and admin-uploaded files with the same name. Now when running manageacl.py, you must specify the full storageUrl whose ACLs require updating.
If you have an existing search index from before this release, please follow steps in README to add the new storageUrl field and backfill it to existing chunks in the index:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#enabling-user-document-upload

What's Changed

• Adds storageURL field to track file location by @pamelafox in #1535
• Disable key-based storage account access in Bicep by @pamelafox in #1518
• Bump idna from 3.6 to 3.7 in /app/backend by @dependabot in #1525
• Bump vite from 4.5.2 to 4.5.3 in /app/frontend by @dependabot in #1488
• Fix gpt-4 vision link by @adamdougal in #1546

New Contributors

• @adamdougal made their first contribution in #1546

Full Changelog: 2024-04-10...2024-04-19

2024-04-10: Graph SDK for auth setup

We are now using msgraph-sdk instead of direct HTTP calls for our scripts that setup Entra application registrations. There should be no functional difference after the change.

What's Changed

• Bump pillow from 10.2.0 to 10.3.0 in /app/backend by @dependabot in #1486
• Port to the Graph SDK for authentication scripts by @pamelafox in #1510

Full Changelog: 2024-03-09...2024-03-10

2024-04-09: Optional user document upload feature

The latest release includes a feature that changes the codebase significantly, the ability for users to upload documents. In order to implement this feature, we moved prepdocslib into app/backend (out of scripts). If your fork has changes in prepdocslib, you may want to merge with the commit before this feature, and then merge this commit, which only moves the files without changing them.

If you want to enable user document upload, follow the documentation at: https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#enabling-user-document-upload
Note that you will need to enable user authentication as a prerequisite for user upload, since only authenticated users can upload.
⚠️ Please see these open issues regarding access control: #1531 , #1532

What's Changed

• Optional upload documents feature by @pamelafox in #1395
• Update README.md with right links to deploy docs by @pamelafox in #1466
• Correct datalakepath argument in prepdocs.sh and prepdocs.ps1 by @pamelafox in #1478
• Bump the github-actions group with 1 update by @dependabot in #1474
• Enforce RBAC only for non-free search service by @pamelafox in #1502
• Update link to pricing calculator to fix Document Intelligence estimation by @pamelafox in #1480
• Update document level access control docs by @mattgotteiner in #1511
• Fix Access control list documentation link issues by @mattgotteiner in #1512

Full Changelog: 2024-03-26...2024-03-09

2024-03-26: New embedding models, more azd environment variables

This release includes additional ways to customize your application setup.

You can try out the new embedding models following steps here:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#using-text-embedding-3-models

You can also use environment variables to reuse existing Azure resources (which previously required a Bicep change):
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_existing.md

Please let us know if you run into any issues with the latest release by filing an issue in the issue tracker.

What's Changed

• Force soft delete and purge for keyvault by @tonybaloney in #1462
• Added Environment Variables for setting Names of Resources by @giterinhub in #1389
• Fix broken links by @pamelafox in #1450
• Add support for using new embedding models with different dimensions by @pamelafox in #1378

Full Changelog: 2024-03-25...2024-03-26

2024-03-25: GPT-4 environment variables

You can now specify your GPT model name and version entirely in the azd environment variables. See updated documentation here:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#using-gpt-4

What's Changed

• Fix search secret name environment variable for free deploys by @pamelafox in #1455
• Allow manual setting of the OpenAI Deployment Name, Version, Capacity and Model by @john0isaac in #1386

Full Changelog: 2024-03-20...2024-03-25

2024-03-20: New settings for minimum search score for search results

This release includes a new feature to restrict the search results based on the minimum score, via a new option in Developer Settings and new POST parameters. You can learn more about search scoring here:
https://learn.microsoft.com/azure/search/hybrid-search-ranking#scores-in-a-hybrid-search-results
You may want to experiment with scoring thresholds if you find that the search results are not always relevant. Reminder that you can use the evaluator tools to see differences in app settings:
https://github.com/Azure-Samples/ai-rag-chat-evaluator

What's Changed

• Update chatreadretrieveread.py (Typo "Azure Open AI"→"Azure OpenAI") by @hyoshioka0128 in #1443
• Update retrievethenread.py (Typo "Azure Open AI"→"Azure OpenAI") by @hyoshioka0128 in #1447
• Bump the github-actions group with 1 update by @dependabot in #1439
• Add minimum score criteria for AI search results by @sogue in #1417

Full Changelog: 2024-03-18...2024-03-20

2024-03-18: Markdown viewer

The code change in this release is the addition of a Markdown viewer, so that users can click on a cited markdown file and see it formatted in the browser.

What's Changed

• Add Markdown File Viewer by @yuvalyaron in #1384
• Added an instruction to Compile the JavaScript, required for pytest to pass. by @giterinhub in #1425
• Update other_samples.md to reflect current features/tech by @pamelafox in #1388
• Update approach.py (Typo "Azure Open AI"→"Azure OpenAI") by @hyoshioka0128 in #1431
• Fix links in documentation by @pamelafox in #1426
• Update gpt4v.md (Typo "Azure Open AI models"→"Azure OpenAI models") by @hyoshioka0128 in #1435

New Contributors

• @giterinhub made their first contribution in #1425

Full Changelog: 2024-03-14...2024-03-18

2024-03-14: AZURE_OPENAI_API_VERSION, bug fixes

This new release includes the ability customize the OpenAI API version via an environment variable (AZURE_OPENAI_API_VERSION) and also updates the version to the latest preview, 2024-03-01-preview. This release also includes an important bugfix in the prepdocs script, so please pull in that fix if you're getting errors with computing embeddings.

What's Changed

• Fix provision for load balance custom by @diberry in #1396
• Update data_ingestion.md with correct notebook link by @pamelafox in #1405
• Bump the github-actions group with 1 update by @dependabot in #1400
• Add needed permissions for template-analyzer workflow by @pamelafox in #1411
• Set Azure OpenAI API version in global configuration by @sogue in #1399
• Add markdown checker by @john0isaac in #1377
• Refactoring of prepdocs for easier integration with user upload feature by @pamelafox in #1407
• Don't disable text chunking when GPT4vision is enabled by @pamelafox in #1355
• Ensure there are no zero-length sections for batch API by @tonybaloney in #1423

New Contributors

• @sogue made their first contribution in #1399

Full Changelog: 2024-03-07...2024-03-14

2024-03-07: Text parser, "Thought process" enhancements

This release includes a simple local parser for text files (including markdown), as well as enhancements to the "Thought process" tab to show additional details about the answer flow, like search results scores and models/deployments used. We aim to add a "score threshold" in the future so that you can filter out results with too low of a score.

What's Changed

• Add search scores, models, and deployments to "Thought process" tab, surface additional properties by @pamelafox in #1375
• Add Markdown/Text Parser by @yuvalyaron in #1381

New Contributors

• @yuvalyaron made their first contribution in #1381

Full Changelog: 2024-03-06...2024-03-07