Add new Azure Flat object store file source

[guerler]

Add a dedicated Azure Flat file source using fsspec adlfs. This adds a new azureflat plugin for Azure Blob Storage accounts without hierarchical namespace. It uses fsspec’s AzureBlobFileSystem, matching the s3fs integration, and avoids the filesystem assumptions in the existing Azure plugin.

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. Create an Azure Storage Account and Containers without Hierarchical Name Spacing
  2. Upload files to the Containers
  3. Update your file_sources_conf.yml

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[bgruening]

@guerler could you please also add object store and file-source templates:

https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/objectstore/templates/examples
https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/files/templates/examples

That would be super helpful I think.

[davelopez]

The implementation looks solid to me! Thank you!

You may want to add adlfs to conditional dependencies too.

[dannon]

Are there mixed flat/hierarchical azure buckets?

edit for a bit more context:

In #21051 that was the problem, there was actually a mix where some keys were 'flat' so to speak (without directory markers), but others were properly hierarchical.