[BUG] az devops extension install on cloudshell container running locally via Docker fails.

[goldjg]

To Reproduce

Dockerfile contains:
FROM mcr.microsoft.com/azure-cloudshell:latest

In running container, enter:
az extension add --name azure-devops

Observed Behavior

root@8a74adb9201d:/usr/cloudshell# az extension add --name azure-devops
An error occurred. Pip failed with status code 1. Use --debug for more information.
root@8a74adb9201d:/usr/cloudshell# az extension add --name azure-devops --debug
cli.knack.cli: Command arguments: ['extension', 'add', '--name', 'azure-devops', '--debug']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7fb4280c7c70>, <function OutputProducer.on_global_arguments at 0x7fb427e2c4c0>, <function CLIQuery.on_global_arguments at 0x7fb427e5d6c0>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'extension': ['azure.cli.command_modules.extension']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: extension                 0.002         1         7
cli.azure.cli.core: Total (1)                 0.002         1         7
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
cli.azure.cli.core: ai-examples               0.146         1         1  /opt/az/lib/python3.10/site-packages/azure-cli-extensions/ai-examples
cli.azure.cli.core: Total (1)                 0.146         1         1  
cli.azure.cli.core: Loaded 2 groups, 8 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : extension add
cli.azure.cli.core: Command table: extension add
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7fb42753f370>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2023-09-20.09-16-55.extension_add.24.log'.     
az_command_data_logger: command args: extension add --name {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7fb42734cdc0>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7fb4273f9240>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7fb4273f9360>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7fb427e2c550>, <function CLIQuery.handle_query_parameter at 0x7fb427e5d750>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7fb4273f92d0>]
urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
urllib3.connectionpool: https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 0
urllib3.connectionpool: Starting new HTTPS connection (1): azcliextensionsync.blob.core.windows.net:443
urllib3.connectionpool: https://azcliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 3213580
cli.azure.cli.core.extension._resolve: Candidates ['azure_devops-0.12.0-py2.py3-none-any.whl', 'azure_devops-0.17.0-py2.py3-none-any.whl', 'azure_devops-0.21.0-py2.py3-none-any.whl', 'azure_devops-0.26.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['azure_devops-0.12.0-py2.py3-none-any.whl', 'azure_devops-0.17.0-py2.py3-none-any.whl', 'azure_devops-0.21.0-py2.py3-none-any.whl', 'azure_devops-0.26.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['azure_devops-0.12.0-py2.py3-none-any.whl', 'azure_devops-0.17.0-py2.py3-none-any.whl', 'azure_devops-0.21.0-py2.py3-none-any.whl', 'azure_devops-0.26.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['azure_devops-0.26.0-py2.py3-none-any.whl', 'azure_devops-0.21.0-py2.py3-none-any.whl', 'azure_devops-0.17.0-py2.py3-none-any.whl', 'azure_devops-0.12.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Choosing the latest of the remaining candidates.
cli.azure.cli.core.extension._resolve: Chosen {'downloadUrl': 'https://github.com/Azure/azure-devops-cli-extension/releases/download/20230127.2/azure_devops-0.26.0-py2.py3-none-any.whl', 'filename': 'azure_devops-0.26.0-py2.py3-none-any.whl', 'metadata': {'azext.minCliCoreVersion': '2.30.0', 'classifiers': ['Development Status :: 4 - Beta', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming 
Language :: Python :: 3.6', 'License :: OSI Approved :: MIT License'], 'extensions': {'python.details': {'contacts': [{'email': 'VSTS_Social@microsoft.com', 'name': 'Microsoft', 'role': 'author'}], 'document_names': {'description': 'DESCRIPTION.rst'}, 'project_urls': {'Home': 'https://github.com/Microsoft/azure-devops-cli-extension'}}}, 'extras': [], 'generator': 'bdist_wheel (0.30.0)', 'license': 'MIT', 'metadata_version': '2.0', 'name': 'azure-devops', 'run_requires': [{'requires': ['distro (==1.3.0)']}], 'summary': 'Tools for managing Azure DevOps.', 'version': '0.26.0'}, 'sha256Digest': '565fc207f1740c26957f382fe2eefabec254011fb2d1b50c0e540f894f47dcbe'}
cli.azure.cli.core.extension.operations: Extension source is url? True
cli.azure.cli.core.extension.operations: Downloading https://github.com/Azure/azure-devops-cli-extension/releases/download/20230127.2/azure_devops-0.26.0-py2.py3-none-any.whl to /tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl
urllib3.connectionpool: Starting new HTTPS connection (1): github.com:443
urllib3.connectionpool: https://github.com:443 "GET /Azure/azure-devops-cli-extension/releases/download/20230127.2/azure_devops-0.26.0-py2.py3-none-any.whl HTTP/1.1" 302 0
urllib3.connectionpool: Starting new HTTPS connection (1): objects.githubusercontent.com:443
urllib3.connectionpool: https://objects.githubusercontent.com:443 "GET /github-production-release-asset-2e65be/107708057/8c8c37f1-25aa-4394-9b37-ab8655c6d437?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230920T091618Z&X-Amz-Expires=300&X-Amz-Signature=c16490d1312d4d029bd0fc895d6327a6e1d9f814aacbe7c3f229794ddd4f161e&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=107708057&response-content-disposition=attachment%3B%20filename%3Dazure_devops-0.26.0-py2.py3-none-any.whl&response-content-type=application%2Foctet-stream HTTP/1.1" 200 1195005
cli.azure.cli.core.extension.operations: Downloaded to /tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl
cli.azure.cli.core.extension.operations: Validating the extension /tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl
cli.azure.cli.core.extension.operations: Checksum of /tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl is OK
cli.azure.cli.core.extension.operations: Validation successful on /tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl
cli.azure.cli.core.extension.operations: Linux distro check: Reading from: /etc/apt/sources.list.d/azure-cli.list
cli.azure.cli.core.extension.operations: Linux distro check: Found in list file: main

cli.azure.cli.core.extension.operations: Linux distro check: Reported by API: None
cli.azure.cli.core.extension.operations: Linux distro check: Mismatch distribution name in /etc/apt/sources.list.d/azure-cli.list file
cli.azure.cli.core.extension.operations: Linux distro check: If command fails, install the appropriate package for your distribution or change the above file accordingly.
cli.azure.cli.core.extension.operations: Linux distro check: /etc/apt/sources.list.d/azure-cli.list has 'main
', current distro is 'None'
cli.azure.cli.core.extension.operations: Executing pip with args: ['install', '--target', '/root/.azure/cliextensions/azure-devops', '/tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension.operations: Running: ['/usr/bin/../../opt/az/bin/python3', '-m', 'pip', 'install', '--target', '/root/.azure/cliextensions/azure-devops', '/tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']
cli.azure.cli.core.extension.operations: Using pip 22.1 from /opt/az/lib/python3.10/site-packages/pip (python 3.10)
Non-user install due to --prefix or --target option
Created temporary directory: /tmp/pip-target-_d3kceq0
Created temporary directory: /tmp/pip-ephem-wheel-cache-dzaqjel5
Created temporary directory: /tmp/pip-build-tracker-vbifnaqa
Initialized build tracking at /tmp/pip-build-tracker-vbifnaqa
Created build tracker: /tmp/pip-build-tracker-vbifnaqa
Entered build tracker: /tmp/pip-build-tracker-vbifnaqa
Created temporary directory: /tmp/pip-install-mfbqhndn
Processing /tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl
  Added azure-devops==0.26.0 from file:///tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl to build tracker '/tmp/pip-build-tracker-vbifnaqa'
  Removed azure-devops==0.26.0 from file:///tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl from build tracker '/tmp/pip-build-tracker-vbifnaqa'
1 location(s) to search for versions of distro:
* https://pypi.org/simple/distro/
Fetching project page and analyzing links: https://pypi.org/simple/distro/
Getting page https://pypi.org/simple/distro/
Found index url https://pypi.org/simple
Starting new HTTPS connection (1): pypi.org:443
Incremented Retry for (url='/simple/distro/'): Retry(total=4, connect=None, read=None, redirect=None, status=None)
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/distro/
Starting new HTTPS connection (2): pypi.org:443
Incremented Retry for (url='/simple/distro/'): Retry(total=3, connect=None, read=None, redirect=None, status=None)
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/distro/
Starting new HTTPS connection (3): pypi.org:443
Incremented Retry for (url='/simple/distro/'): Retry(total=2, connect=None, read=None, redirect=None, status=None)
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/distro/
Starting new HTTPS connection (4): pypi.org:443
Incremented Retry for (url='/simple/distro/'): Retry(total=1, connect=None, read=None, redirect=None, status=None)
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/distro/
Starting new HTTPS connection (5): pypi.org:443
Incremented Retry for (url='/simple/distro/'): Retry(total=0, connect=None, read=None, redirect=None, status=None)
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/distro/
Starting new HTTPS connection (6): pypi.org:443
Could not fetch URL https://pypi.org/simple/distro/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/distro/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))) - skipping
Skipping link: not a file: https://pypi.org/simple/distro/
Given no hashes to check 0 links for project 'distro': discarding no candidates
ERROR: Could not find a version that satisfies the requirement distro==1.3.0 (from azure-devops) (from versions: none)
ERROR: No matching distribution found for distro==1.3.0
Exception information:
Traceback (most recent call last):
  File "/opt/az/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 94, in resolve
    result = self._result = resolver.resolve(
  File "/opt/az/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/opt/az/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 385, in resolve
    raise ResolutionImpossible(self.state.backtrack_causes)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('distro==1.3.0'), parent=LinkCandidate('file:///tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl'))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/az/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/opt/az/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/opt/az/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 340, in run
    requirement_set = resolver.resolve(
  File "/opt/az/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 103, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: No matching distribution found for distro==1.3.0
Removed build tracker: '/tmp/pip-build-tracker-vbifnaqa'

cli.azure.cli.core.extension.operations: Command '['/usr/bin/../../opt/az/bin/python3', '-m', 'pip', 'install', '--target', '/root/.azure/cliextensions/azure-devops', '/tmp/tmpgq20qdat/azure_devops-0.26.0-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at /root/.azure/cliextensions/azure-devops
cli.azure.cli.core.util: azure.cli.core.util.handle_exception is called with an exception:
cli.azure.cli.core.util: Traceback (most recent call last):
  File "/opt/az/lib/python3.10/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
    return add_extension(cli_ctx=cmd.cli_ctx, source=source, extension_name=extension_name, index_url=index_url,
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 341, in add_extension
    extension_name = _add_whl_ext(cli_ctx=cmd_cli_ctx, source=source, ext_sha256=ext_sha256,
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 173, in _add_whl_ext
    raise CLIError('An error occurred. Pip failed with status code {}. '
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7fb42753f5b0>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 42.266 seconds (init: 0.115, invoke: 42.152)
telemetry.save: Save telemetry record of length 2991 in cache
telemetry.check: Negative: The /root/.azure/telemetry.txt was modified at 2023-09-20 09:16:50.984047, which in less than 600.000000 s
root@8a74adb9201d:/usr/cloudshell# less /root/.azure/telemetry.txt
root@8a74adb9201d:/usr/cloudshell# 

Expected behavior

I expected the devops extension to cleanly install.

Is this specific to Cloud Shell?

Yes

Interface information

Docker container, image built from dockerfile which pulls from mcr.microsoft.com/azure-cloudshell:latest
Docker Desktop is running on a Windows 10 laptop

Additional context

Add any other context about the problem here.

[goldjg]

I do see SSL errors - however pypi should be using same SSL certs as Python? If it did it would work - I had to include our proxy certs in the cert bundle to allow az login etc to work (e.g. enable python to do SSL)

[theJasonHelmick]

Hi @goldjg - thank you for reporting this issue. We recently began releasing a container image of Azure's Cloud Shell and appreciate you reporting this as we improve the functionality of our container release. I was unaware of this issue and have had different experiences - so we will need to investigate this further. I don't have a timeline yet, but I will respond in this issue. Thank you!