Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QCPortal 0.56 not available on Apple Silicon #846

Closed
mattwthompson opened this issue Jul 23, 2024 · 9 comments
Closed

QCPortal 0.56 not available on Apple Silicon #846

mattwthompson opened this issue Jul 23, 2024 · 9 comments

Comments

@mattwthompson
Copy link
Contributor

Describe the bug

I cannot install QCPortal 0.56 natively on Apple Silicon (M1/M2/etc.) but I can if emulating Intel hardware.

To Reproduce

$ micromamba create --name qcportal-m1 "qcportal =0.56" -c conda-forge
warning  libmamba 'root_prefix' set with default value: /Users/mattthompson/micromamba
conda-forge/osx-arm64                               10.8MB @  11.7MB/s  1.0s
conda-forge/noarch                                 @  15.9MB/s  1.0s
error    libmamba Could not solve for environment specs
    The following package could not be installed
    └─ qcportal 0.56**  is not installable because it requires
       └─ apsw, which does not exist (perhaps a missing channel).
critical libmamba Could not solve for environment specs
(openff-interchange-dev) [openff-qcsubmit]
$ CONDA_SUBDIR=osx-64 micromamba create --name qcportal-m1 "qcportal =0.56" -c conda-forge
warning  libmamba 'root_prefix' set with default value: /Users/mattthompson/micromamba
conda-forge/noarch                                          Using cache
conda-forge/osx-64                                  32.1MB @  33.9MB/s  0.9s

Transaction

  Prefix: /Users/mattthompson/micromamba/envs/qcportal-m1

  Updating specs:

   - qcportal=0.56


  Package                Version  Build               Channel           Size
──────────────────────────────────────────────────────────────────────────────
  Install:
──────────────────────────────────────────────────────────────────────────────

  + python_abi              3.12  4_cp312             conda-forge        6kB
  + ncurses                  6.5  h5846eda_0          conda-forge      824kB
  + xz                     5.2.6  h775f41a_0          conda-forge      238kB
  + libexpat               2.6.2  h73e2aa4_0          conda-forge       69kB
  + libzlib                1.3.1  h87427d6_1          conda-forge       57kB
  + yaml                   0.2.5  h0d85af4_2          conda-forge       84kB
  + bzip2                  1.0.8  hfdf4475_7          conda-forge      134kB
  + libffi                 3.4.2  h0d85af4_5          conda-forge       51kB
  + libcxx                18.1.8  hef8daea_0          conda-forge        1MB
  + ca-certificates     2024.7.4  h8857fd0_0          conda-forge      154kB
  + llvm-openmp           18.1.8  h15ab845_0          conda-forge      301kB
  + readline                 8.2  h9e318b2_1          conda-forge      256kB
  + tk                    8.6.13  h1abcd95_1          conda-forge        3MB
  + zstd                   1.5.6  h915ae27_0          conda-forge      499kB
  + libsqlite             3.46.0  h1b8f9f3_0          conda-forge      909kB
  + openssl                3.3.1  h87427d6_2          conda-forge        3MB
  + libgfortran5          13.2.0  h2873a65_3          conda-forge        2MB
  + sqlite                3.46.0  h28673e1_0          conda-forge      912kB
  + libgfortran            5.0.0  13_2_0_h97931a8_3   conda-forge      110kB
  + libopenblas           0.3.27  openmp_h8869122_1   conda-forge        6MB
  + libblas                3.9.0  22_osx64_openblas   conda-forge       15kB
  + libcblas               3.9.0  22_osx64_openblas   conda-forge       15kB
  + liblapack              3.9.0  22_osx64_openblas   conda-forge       15kB
  + tzdata                 2024a  h0c530f3_0          conda-forge     Cached
  + python                3.12.4  h37a9e06_0_cpython  conda-forge       14MB
  + wheel                 0.43.0  pyhd8ed1ab_1        conda-forge     Cached
  + setuptools            71.0.4  pyhd8ed1ab_0        conda-forge        1MB
  + pip                     24.0  pyhd8ed1ab_0        conda-forge     Cached
  + hyperframe             6.0.1  pyhd8ed1ab_0        conda-forge     Cached
  + charset-normalizer     3.3.2  pyhd8ed1ab_0        conda-forge     Cached
  + hpack                  4.0.0  pyh9f0ad1d_0        conda-forge     Cached
  + pysocks                1.7.1  pyha2e5f31_6        conda-forge     Cached
  + idna                     3.7  pyhd8ed1ab_0        conda-forge     Cached
  + certifi             2024.7.4  pyhd8ed1ab_0        conda-forge     Cached
  + colorama               0.4.6  pyhd8ed1ab_0        conda-forge     Cached
  + pycparser               2.22  pyhd8ed1ab_0        conda-forge     Cached
  + python-tzdata         2024.1  pyhd8ed1ab_0        conda-forge     Cached
  + six                   1.16.0  pyh6c4a22f_0        conda-forge     Cached
  + appdirs                1.4.4  pyh9f0ad1d_0        conda-forge       13kB
  + typing_extensions     4.12.2  pyha770c72_0        conda-forge     Cached
  + tabulate               0.9.0  pyhd8ed1ab_1        conda-forge     Cached
  + pytz                  2024.1  pyhd8ed1ab_0        conda-forge     Cached
  + pyjwt                  2.8.0  pyhd8ed1ab_1        conda-forge     Cached
  + packaging               24.1  pyhd8ed1ab_0        conda-forge     Cached
  + pycalverter            1.6.1  py_0                conda-forge       15kB
  + h2                     4.1.0  pyhd8ed1ab_0        conda-forge     Cached
  + tqdm                  4.66.4  pyhd8ed1ab_0        conda-forge     Cached
  + python-dateutil        2.9.0  pyhd8ed1ab_0        conda-forge     Cached
  + typing-extensions     4.12.2  hd8ed1ab_0          conda-forge     Cached
  + annotated-types        0.7.0  pyhd8ed1ab_0        conda-forge     Cached
  + flexparser             0.3.1  pyhd8ed1ab_0        conda-forge       29kB
  + flexcache                0.3  pyhd8ed1ab_0        conda-forge       17kB
  + pint                  0.24.3  pyhd8ed1ab_0        conda-forge      229kB
  + brotli-python          1.1.0  py312heafc425_1     conda-forge      367kB
  + numpy                  2.0.0  py312h8813227_0     conda-forge        7MB
  + msgpack-python         1.0.8  py312hc3c9ca0_0     conda-forge       92kB
  + pyyaml                 6.0.1  py312h104f124_1     conda-forge      186kB
  + cffi                  1.16.0  py312h38bf5a0_0     conda-forge      282kB
  + apsw                3.46.0.0  py312h77d72a6_0     conda-forge      535kB
  + pydantic-core         2.20.1  py312ha47ea1c_0     conda-forge        2MB
  + pandas                 2.2.2  py312h1171441_1     conda-forge       15MB
  + zstandard             0.23.0  py312h331e495_0     conda-forge      411kB
  + pydantic               2.8.2  pyhd8ed1ab_0        conda-forge     Cached
  + urllib3                2.2.2  pyhd8ed1ab_1        conda-forge     Cached
  + qcelemental           0.28.0  pyhd8ed1ab_0        conda-forge     Cached
  + requests              2.32.3  pyhd8ed1ab_0        conda-forge     Cached
  + qcportal                0.56  pyhd8ed1ab_0        conda-forge       84kB

  Summary:

  Install: 67 packages

  Total download: 61MB

──────────────────────────────────────────────────────────────────────────────


Confirm changes: [Y/n]

Expected behavior

I expected to be able to install the most recent release of QCPortal or find documentation of this limitation.

Additional context

This appears to be related to the apsw refactor (#829). There are no osx-arm64 builds available. Upstream solutions are multiple months and several changes away.

I couldn't find any documentation on this using a few similar search terms. I mostly following the installation documentation; micromamba vs. conda shouldn't make a difference here. Apologies if I missed something

@j-wags
Copy link

j-wags commented Jul 24, 2024

Just to add some context - This appears to be a regression, as OSX-ARM-compatible packages were available for as high as QC* 0.54, but aren't available for 0.55 or 0.56.

@loriab
Copy link
Collaborator

loriab commented Jul 24, 2024

On apsw compatibility, what are you reading as several months away for upstream: (a) aarch64, (b) osx-arm64 for >=py310, (c) osx-arm64 for <py310 ? I couldn't tell. If (b) is good upstream and just (a) or (c) are lacking, it might be fixable c-f side. I hadn't noticed this on psi4 since our Mac GHA runner is Intel (and broken for other reasons :-( ).

@bennybp
Copy link
Contributor

bennybp commented Jul 24, 2024

This issue from a few years ago seems to hint that it should be possible to build on ARM, but I don't have a Mac or much experience there: rogerbinns/apsw#353

In that case, conda-forge just needs to be updated? Unfortunately I am no help there

@loriab
Copy link
Collaborator

loriab commented Jul 24, 2024

It might be ok on silicon:

python setup.py test
running test
                Python  /Users/loriab/linux/miniconda-silicon/envs/p4dev312illegal/bin/python sys.version_info(major=3, minor=12, micro=4, releaselevel='final', serial=0) 64bit 
Testing with APSW file  /Users/loriab/linux/miniconda-silicon/envs/p4dev312illegal/lib/python3.12/site-packages/apsw/__init__.cpython-312-darwin.so
          APSW version  3.46.0.1
    SQLite lib version  3.46.0
SQLite headers version  3046000
    Using amalgamation  False
Not doing the optional LoadExtension test.  You need to compile the extension first

  python3 setup.py build_test_extension
..............................................................A message due to RecursionError is possible, and what is being tested
object address  : 0x104f57b80
object refcount : 1
object type     : 0x100da3ed8
object type name: TypeError
object repr     : 
lost sys.stderr
..................................................................
----------------------------------------------------------------------
Ran 128 tests in 26.262s

OK

It compiled something, though I'm not sure it tested.

@mattwthompson
Copy link
Contributor Author

I linked a comment about Linux ARM, purely my mistake. It is a supported platform on apsw. This should be fixable with an upstream migration in the feedstock: conda-forge/apsw-feedstock#57

I don't what platforms QCArchive is meant to support - as downstream users, we were using it on M1 until 0.55. I'm actually confused on how it's supposed to be installed - the docs recommend installing using conda-forge but say the packages exist on the QCArchive channel, which doesn't seem to have recent versions - or should I not be using Conda infrastructure at all?

FWIW, GitHub provides (free) M1 runners for use in CI, which might have caught this in development (are tests meant to only run on Ubuntu?)

@loriab
Copy link
Collaborator

loriab commented Jul 24, 2024

I just opened the migration request conda-forge/conda-forge-pinning-feedstock#6200 . It'll probably need python restrictions in the feedstock.

afaik, the QCArchive conda channel was a testbed for next packages. Psi4 has been getting qcf from main c-f for a while now. I think qcf is meant to support anything that the deps allow -- that ruled out windows that I knew about. Ben would have to speak to pythons.

@loriab
Copy link
Collaborator

loriab commented Jul 25, 2024

Ok the osx-arm64 PR is ready at conda-forge/apsw-feedstock#58 . It's cross-compiling so there's no testing, but hopefully it's ok. That should close this issue. I'll give the feedstock maintainer a couple days to look it over before merging.

Since all the key pydantic parties are on this thread, I'll briefly hijack it to point out https://docs.pydantic.dev/latest/migration/#using-pydantic-v1-features-in-a-v1v2-environment which says with the newest pyd v1 (1.10.17), they've unified the v1 namespace so one can depend on >=1.10.17 and use a unified from pydantic.v1.<module> import <object> rather than try/v2/except/v1 block. Might be worth moving to for the (hopefully) short remaining period of v1-only.

@loriab
Copy link
Collaborator

loriab commented Jul 29, 2024

Looks like osx-arm64 packages are up on anaconda.org, though likely not mirrored yet.

@mattwthompson
Copy link
Contributor Author

All systems go from what I can see right now - thanks for the quick action @loriab!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants