ModuleNotFoundError: import of fnmatch halted; None in sys.modules

[optroodt]

I'm trying to use records with Snowflake (dwh solution in the cloud), and I do get results but I always get the ModuleNotFoundError error.

It works without any problems when I use just SqlAlchemy using snowflake-sqlalchemy, so that makes me think it's something records specific. I really like the product and, although my resources are limited, I'm willing to assist or provide more information if requested.

I'm running Python 3.7.2, installed through homebrew, in a virtualenv on MacOS 10.14.3.

Installed packages:

pip install records snowflake-sqlalchemy

Code to reproduce (credentials and account are obviously not the ones I use):

import records
from sqlalchemy import create_engine

connection_params = dict(
    user='user',
    password='password',
    account='abc123',
    database='DWH',
    schema='SCHEMA'
)

connection_string = 'snowflake://{user}:{password}@{account}/{database}/{schema}'
db_string = connection_string.format(**connection_params)

query = 'select * from table limit 10'

db = records.Database(db_string)
rows = db.query(query)
for row in rows:
    print(row)

Stacktrace

Exception during reset or similar
Traceback (most recent call last):
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/sqlalchemy/pool.py", line 742, in _finalize_fairy
    fairy._reset(pool)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/sqlalchemy/pool.py", line 929, in _reset
    pool._dialect.do_rollback(self)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 486, in do_rollback
    dbapi_connection.rollback()
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/connection.py", line 508, in rollback
    self.cursor().execute("ROLLBACK")
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/cursor.py", line 479, in execute
    _is_put_get=_is_put_get)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/cursor.py", line 369, in _execute_helper
    _no_results=_no_results)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/connection.py", line 784, in cmd_query
    _include_retry_params=True)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/network.py", line 313, in request
    timeout=timeout, _include_retry_params=_include_retry_params)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/network.py", line 530, in _post_request
    _include_retry_params=_include_retry_params)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/network.py", line 608, in fetch
    **kwargs)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/network.py", line 685, in _request_exec_wrapper
    raise e
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/network.py", line 648, in _request_exec_wrapper
    **kwargs)
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/network.py", line 873, in _request_exec
    raise err
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/snowflake/connector/network.py", line 776, in _request_exec
    auth=SnowflakeAuth(token),
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/botocore/vendored/requests/sessions.py", line 456, in request
    prep.url, proxies, stream, verify, cert
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/botocore/vendored/requests/sessions.py", line 614, in merge_environment_settings
    env_proxies = get_environ_proxies(url) or {}
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/botocore/vendored/requests/utils.py", line 534, in get_environ_proxies
    if should_bypass_proxies(url):
  File "/Users/optroodt/virtualenvs/records/lib/python3.7/site-packages/botocore/vendored/requests/utils.py", line 523, in should_bypass_proxies
    bypass = proxy_bypass(netloc)
  File "/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2610, in proxy_bypass
    return proxy_bypass_macosx_sysconf(host)
  File "/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2587, in proxy_bypass_macosx_sysconf
    return _proxy_bypass_macosx_sysconf(host, proxy_settings)
  File "/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 2534, in _proxy_bypass_macosx_sysconf
    from fnmatch import fnmatch
ModuleNotFoundError: import of fnmatch halted; None in sys.modules

Thanks!

[sryza]

I'm running into this as well. Did you figure out a solution?

[optroodt]

No, I didn't. I set the project aside for a bit. What's your environment (os, python etc)?

[sryza]

OS: macOS Mojave 10.14.2
Python: 3.7.0

[lsatterfield]

I found this thread googling the same error. I believe my issue was related to some missing under scores.

I had defined a class

class table(base):
tablename = 'table'
__table_args = {'schema': 'schema',
'autoload': True, 'autoload_with': engine}
and when the class is defined it couldn't find the __table_args

adding the underscores to __table_args -> __table_args__ fixed my issue.

[timofurrer]

@optroodt @sryza Can you reproduce @lsatterfield's fix?

[lsatterfield]

I seem to have made a mistake in my earlier post. I didnt realize it would bold if i had leading and trailing underscores. Should be __table_args -> __table_args__ .

-- Edited Above for accuracy, thanks timofurrer

[timofurrer]

You can put the code in backticks. See https://help.github.com/en/articles/creating-and-highlighting-code-blocks

(you can edit your post above) 🎉

[sryza]

If I understand correctly, I don't think @lsatterfield's fix applies in my case. All I'm doing is

    sqlalchemy_cols = [Column(col_name, data_type, comment=comment) for col_name, data_type, comment in columns]
    metadata = MetaData(bind=con)
    table = Table(table_name, metadata, *sqlalchemy_cols, schema=schema)
    table.create()

where con is a connection from a snowflake SQLAlchemy engine.

[optroodt]

I also don't think @lsatterfield 's fix is related. I'm not configuring anything in SQLAlchemy, just running a plain query.

I can say that it works without problems under Python 3.6.5.

[peterolive]

@optroodt
You need to close the connection and dispose the engine.
Try add the following at the end:

connection.close()
engine.dispose()

[ynux]

I'm experiencing the same issue, running into this when I do python xxx.py. Note that I log in with two factor auth and have to log in twice (once for the connection, once for create table). Confusingly, it works when i define a function and then call that function, or when I run the code in an interactive shell. (Python 3.7.4)

[samisken]

@peterolive thanks! This solved my issue with this error

[amodig]

Probable root cause in the Snowflake's Python connector:
https://community.snowflake.com/s/article/Python-connector-does-not-implement-a-destructor-method-correctly