SSL: CERTIFICATE_VERIFY_FAILED when get_object s3

[ctran-PLN]

Spec MacOs:
python 3.11.4
botocore 1.34.55

If i use my AWS CLI to call s3 like this aws s3 ls i will get results.
however, when I do it through python botocore3

import botocore
import botocore.session
    
session = botocore.session.get_session()
s3 = session.create_client('s3')
s3.get_object(Bucket=bucket_name, Key=object_key)

I get

2024-03-06 20:19:12,955 botocore.httpsession [DEBUG] Certificate path: /Users/user/venv-dds/lib/python3.11/site-packages/certifi/cacert.pem
2024-03-06 20:19:12,956 urllib3.connectionpool [DEBUG] Starting new HTTPS connection (14): s3-dev.s3.amazonaws.com:443
2024-03-06 20:19:14,269 botocore.endpoint [DEBUG] Exception received when sending HTTP request.
Traceback (most recent call last):
  File "/Users/user/venv-dds/lib/python3.11/site-packages/urllib3/connectionpool.py", line 468, in _make_request
    self._validate_conn(conn)
  File "/Users/user/venv-dds/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1097, in _validate_conn
    conn.connect()
  File "/Users/user/venv-dds/lib/python3.11/site-packages/urllib3/connection.py", line 642, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/venv-dds/lib/python3.11/site-packages/urllib3/connection.py", line 783, in _ssl_wrap_socket_and_match_hostname
    ssl_sock = ssl_wrap_socket(
               ^^^^^^^^^^^^^^^^
  File "/Users/user/venv-dds/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 471, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/venv-dds/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 515, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.pyenv/versions/3.11.4/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.pyenv/versions/3.11.4/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/Users/user/.pyenv/versions/3.11.4/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1002)[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1002)

[RyanFitzSimmonsAK]

Hey @ctran-PLN, I wasn't able to reproduce this. Could you tell me a bit more about how you have your environment configured? Are you using a proxy? I'd also recommend taking a look at this documentation. It does mention self-signed certificates as one cause, so you might need to use --ca-bundle or an equivalent setting (environment variable or configuration file).

[ctran-PLN]

Hi Ryan, thanks for responding.
It was a drift in my local setup. Somehow the aws cli was setup to use the right certificate while my boto3 library was not. After fixed the issue with our organization devops team, I don't get the error anymore.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.