docker compose up failed in mac Apple M2 Sonoma 14.6

[thetruechar]

 => ERROR [notebooks stage-0  7/22] RUN --mount=type=cache,target=/var/cache/pip     python -m pip install --upgrade pip &&  173.0s
------
 > [notebooks stage-0  7/22] RUN --mount=type=cache,target=/var/cache/pip     python -m pip install --upgrade pip &&     pip install -r requirements.txt &&     python -m spacy download en_core_web_sm:
0.376 Requirement already satisfied: pip in /opt/conda/lib/python3.10/site-packages (23.0)
2.919 Collecting pip
2.928   Using cached pip-24.2-py3-none-any.whl (1.8 MB)
3.446 Installing collected packages: pip
3.446   Attempting uninstall: pip
3.447     Found existing installation: pip 23.0
3.448     Uninstalling pip-23.0:
3.699       Successfully uninstalled pip-23.0
4.183 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
4.183 Successfully installed pip-24.2
4.445 Collecting en-coreference-web-trf==3.4.0a2 (from -r requirements.txt (line 23))
8.069   Downloading https://github.com/explosion/spacy-experimental/releases/download/v0.6.1/en_coreference_web_trf-3.4.0a2-py3-none-any.whl (490.3 MB)
152.1      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 490.3/490.3 MB 3.8 MB/s eta 0:00:00
153.5 Collecting adjusttext==0.8 (from -r requirements.txt (line 1))
153.5   Using cached adjustText-0.8-py3-none-any.whl.metadata (402 bytes)
153.8 Collecting appnope==0.1.0 (from -r requirements.txt (line 2))
153.8   Using cached appnope-0.1.0-py2.py3-none-any.whl.metadata (685 bytes)
154.1 Collecting bs4==0.0.1 (from -r requirements.txt (line 3))
154.1   Using cached bs4-0.0.1.tar.gz (1.1 kB)
154.1   Preparing metadata (setup.py): started
154.2   Preparing metadata (setup.py): finished with status 'done'
154.5 Collecting datasets==2.3.2 (from -r requirements.txt (line 4))
154.5   Using cached datasets-2.3.2-py3-none-any.whl.metadata (20 kB)
154.8 Collecting findspark==1.3.0 (from -r requirements.txt (line 5))
154.8   Using cached findspark-1.3.0-py2.py3-none-any.whl.metadata (315 bytes)
154.8 Requirement already satisfied: ipython-genutils==0.2.0 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 6)) (0.2.0)
155.1 Collecting jupyter-console==6.4.0 (from -r requirements.txt (line 7))
155.1   Using cached jupyter_console-6.4.0-py3-none-any.whl.metadata (951 bytes)
155.5 Collecting lxml==4.9.1 (from -r requirements.txt (line 8))
155.5   Using cached lxml-4.9.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl.metadata (3.3 kB)
155.9 Collecting matplotlib==3.5.2 (from -r requirements.txt (line 9))
155.9   Using cached matplotlib-3.5.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (6.7 kB)
156.2 Collecting mergedeep==1.3.4 (from -r requirements.txt (line 10))
156.2   Using cached mergedeep-1.3.4-py3-none-any.whl.metadata (4.3 kB)
156.6 Collecting mistune==2.0.4 (from -r requirements.txt (line 11))
156.6   Using cached mistune-2.0.4-py2.py3-none-any.whl.metadata (1.5 kB)
156.9 Collecting mizani==0.9.2 (from -r requirements.txt (line 12))
156.9   Using cached mizani-0.9.2-py3-none-any.whl.metadata (4.6 kB)
157.2 Collecting nltk==3.6.6 (from -r requirements.txt (line 13))
157.2   Using cached nltk-3.6.6-py3-none-any.whl.metadata (2.9 kB)
157.7 Collecting nmslib==2.1.1 (from -r requirements.txt (line 14))
157.7   Using cached nmslib-2.1.1.tar.gz (188 kB)
157.7   Preparing metadata (setup.py): started
159.3   Preparing metadata (setup.py): finished with status 'done'
160.2 Collecting plotly==4.14.3 (from -r requirements.txt (line 15))
160.2   Using cached plotly-4.14.3-py2.py3-none-any.whl.metadata (7.6 kB)
160.5 Collecting plotnine==0.10.1 (from -r requirements.txt (line 16))
160.5   Using cached plotnine-0.10.1-py3-none-any.whl.metadata (7.3 kB)
161.8 Collecting prometheus-client==0.11.0 (from -r requirements.txt (line 17))
161.8   Using cached prometheus_client-0.11.0-py2.py3-none-any.whl.metadata (19 kB)
162.1 Collecting py4j==0.10.9.7 (from -r requirements.txt (line 18))
162.1   Using cached py4j-0.10.9.7-py2.py3-none-any.whl.metadata (1.5 kB)
162.8 Collecting pysolr==3.9.0 (from -r requirements.txt (line 19))
162.8   Using cached pysolr-3.9.0.tar.gz (55 kB)
162.9   Preparing metadata (setup.py): started
164.4   Preparing metadata (setup.py): finished with status 'done'
164.7 Collecting qtconsole==5.1.1 (from -r requirements.txt (line 20))
164.7   Using cached qtconsole-5.1.1-py3-none-any.whl.metadata (5.0 kB)
164.9 Collecting retrying==1.3.3 (from -r requirements.txt (line 21))
164.9   Using cached retrying-1.3.3.tar.gz (10 kB)
165.0   Preparing metadata (setup.py): started
165.1   Preparing metadata (setup.py): finished with status 'done'
165.4 Collecting sentence-transformers==2.2.2 (from -r requirements.txt (line 24))
165.4   Using cached sentence-transformers-2.2.2.tar.gz (85 kB)
165.4   Preparing metadata (setup.py): started
165.6   Preparing metadata (setup.py): finished with status 'done'
165.8 Collecting staticmap==0.5.7 (from -r requirements.txt (line 25))
165.8   Using cached staticmap-0.5.7-py3-none-any.whl.metadata (459 bytes)
166.1 Collecting openai-clip==1.0.1 (from -r requirements.txt (line 26))
166.1   Using cached openai-clip-1.0.1.tar.gz (1.4 MB)
166.1   Preparing metadata (setup.py): started
166.3   Preparing metadata (setup.py): finished with status 'done'
166.7 Collecting setuptools==69.5.1 (from -r requirements.txt (line 27))
166.7   Using cached setuptools-69.5.1-py3-none-any.whl.metadata (6.2 kB)
166.9 Collecting spladerunner==0.1.5 (from -r requirements.txt (line 28))
166.9   Using cached SPLADERunner-0.1.5-py3-none-any.whl.metadata (5.1 kB)
167.0 Requirement already satisfied: numpy==1.23.5 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 29)) (1.23.5)
167.0 Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.10/site-packages (from bs4==0.0.1->-r requirements.txt (line 3)) (4.11.2)
167.0 Requirement already satisfied: pyarrow>=6.0.0 in /opt/conda/lib/python3.10/site-packages (from datasets==2.3.2->-r requirements.txt (line 4)) (11.0.0)
167.3 Collecting dill<0.3.6 (from datasets==2.3.2->-r requirements.txt (line 4))
167.3   Using cached dill-0.3.5.1-py2.py3-none-any.whl.metadata (9.7 kB)
167.3 Requirement already satisfied: pandas in /opt/conda/lib/python3.10/site-packages (from datasets==2.3.2->-r requirements.txt (line 4)) (1.5.3)
167.3 Requirement already satisfied: requests>=2.19.0 in /opt/conda/lib/python3.10/site-packages (from datasets==2.3.2->-r requirements.txt (line 4)) (2.28.2)
167.3 Requirement already satisfied: tqdm>=4.62.1 in /opt/conda/lib/python3.10/site-packages (from datasets==2.3.2->-r requirements.txt (line 4)) (4.64.1)
167.6 Collecting xxhash (from datasets==2.3.2->-r requirements.txt (line 4))
167.6   Using cached xxhash-3.4.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (12 kB)
168.0 Collecting multiprocess (from datasets==2.3.2->-r requirements.txt (line 4))
168.0   Using cached multiprocess-0.70.16-py310-none-any.whl.metadata (7.2 kB)
168.0 Requirement already satisfied: fsspec>=2021.05.0 in /opt/conda/lib/python3.10/site-packages (from fsspec[http]>=2021.05.0->datasets==2.3.2->-r requirements.txt (line 4)) (2023.1.0)
168.4 Collecting aiohttp (from datasets==2.3.2->-r requirements.txt (line 4))
168.4   Using cached aiohttp-3.10.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (7.5 kB)
168.8 Collecting huggingface-hub<1.0.0,>=0.1.0 (from datasets==2.3.2->-r requirements.txt (line 4))
168.8   Using cached huggingface_hub-0.24.5-py3-none-any.whl.metadata (13 kB)
168.8 Requirement already satisfied: packaging in /opt/conda/lib/python3.10/site-packages (from datasets==2.3.2->-r requirements.txt (line 4)) (23.0)
169.1 Collecting responses<0.19 (from datasets==2.3.2->-r requirements.txt (line 4))
169.1   Using cached responses-0.18.0-py3-none-any.whl.metadata (29 kB)
169.1 Requirement already satisfied: jupyter-client in /opt/conda/lib/python3.10/site-packages (from jupyter-console==6.4.0->-r requirements.txt (line 7)) (8.0.2)
169.1 Requirement already satisfied: ipython in /opt/conda/lib/python3.10/site-packages (from jupyter-console==6.4.0->-r requirements.txt (line 7)) (8.10.0)
169.1 Requirement already satisfied: ipykernel in /opt/conda/lib/python3.10/site-packages (from jupyter-console==6.4.0->-r requirements.txt (line 7)) (6.21.2)
169.1 Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from jupyter-console==6.4.0->-r requirements.txt (line 7)) (3.0.36)
169.1 Requirement already satisfied: pygments in /opt/conda/lib/python3.10/site-packages (from jupyter-console==6.4.0->-r requirements.txt (line 7)) (2.14.0)
169.1 Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.10/site-packages (from matplotlib==3.5.2->-r requirements.txt (line 9)) (0.11.0)
169.1 Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.10/site-packages (from matplotlib==3.5.2->-r requirements.txt (line 9)) (4.38.0)
169.1 Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib==3.5.2->-r requirements.txt (line 9)) (1.4.4)
169.1 Requirement already satisfied: pillow>=6.2.0 in /opt/conda/lib/python3.10/site-packages (from matplotlib==3.5.2->-r requirements.txt (line 9)) (9.4.0)
169.1 Requirement already satisfied: pyparsing>=2.2.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib==3.5.2->-r requirements.txt (line 9)) (3.0.9)
169.1 Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/lib/python3.10/site-packages (from matplotlib==3.5.2->-r requirements.txt (line 9)) (2.8.2)
169.2 Requirement already satisfied: scipy>=1.5.0 in /opt/conda/lib/python3.10/site-packages (from mizani==0.9.2->-r requirements.txt (line 12)) (1.10.0)
169.2 Requirement already satisfied: click in /opt/conda/lib/python3.10/site-packages (from nltk==3.6.6->-r requirements.txt (line 13)) (8.1.3)
169.2 Requirement already satisfied: joblib in /opt/conda/lib/python3.10/site-packages (from nltk==3.6.6->-r requirements.txt (line 13)) (1.2.0)
169.7 Collecting regex>=2021.8.3 (from nltk==3.6.6->-r requirements.txt (line 13))
169.7   Using cached regex-2024.7.24-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (40 kB)
170.0 Collecting pybind11<2.6.2 (from nmslib==2.1.1->-r requirements.txt (line 14))
170.0   Using cached pybind11-2.6.1-py2.py3-none-any.whl.metadata (8.7 kB)
170.0 Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from nmslib==2.1.1->-r requirements.txt (line 14)) (5.9.4)
170.0 Requirement already satisfied: six in /opt/conda/lib/python3.10/site-packages (from plotly==4.14.3->-r requirements.txt (line 15)) (1.16.0)
170.0 Requirement already satisfied: patsy>=0.5.1 in /opt/conda/lib/python3.10/site-packages (from plotnine==0.10.1->-r requirements.txt (line 16)) (0.5.3)
170.0 Requirement already satisfied: statsmodels>=0.13.2 in /opt/conda/lib/python3.10/site-packages (from plotnine==0.10.1->-r requirements.txt (line 16)) (0.13.5)
170.0 Requirement already satisfied: traitlets in /opt/conda/lib/python3.10/site-packages (from qtconsole==5.1.1->-r requirements.txt (line 20)) (5.9.0)
170.0 Requirement already satisfied: jupyter-core in /opt/conda/lib/python3.10/site-packages (from qtconsole==5.1.1->-r requirements.txt (line 20)) (5.2.0)
170.3 Collecting qtpy (from qtconsole==5.1.1->-r requirements.txt (line 20))
170.3   Using cached QtPy-2.4.1-py3-none-any.whl.metadata (12 kB)
170.3 Requirement already satisfied: pyzmq>=17.1 in /opt/conda/lib/python3.10/site-packages (from qtconsole==5.1.1->-r requirements.txt (line 20)) (25.0.0)
170.7 Collecting transformers<5.0.0,>=4.6.0 (from sentence-transformers==2.2.2->-r requirements.txt (line 24))
170.7   Using cached transformers-4.44.0-py3-none-any.whl.metadata (43 kB)
171.1 Collecting torch>=1.6.0 (from sentence-transformers==2.2.2->-r requirements.txt (line 24))
171.1   Using cached torch-2.4.0-cp310-cp310-manylinux2014_aarch64.whl.metadata (26 kB)
171.4 Collecting torchvision (from sentence-transformers==2.2.2->-r requirements.txt (line 24))
171.4   Using cached torchvision-0.19.0-cp310-cp310-manylinux2014_aarch64.whl.metadata (6.0 kB)
171.4 Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.10/site-packages (from sentence-transformers==2.2.2->-r requirements.txt (line 24)) (1.2.1)
171.7 Collecting sentencepiece (from sentence-transformers==2.2.2->-r requirements.txt (line 24))
171.7   Using cached sentencepiece-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (7.7 kB)
172.0 Collecting ftfy (from openai-clip==1.0.1->-r requirements.txt (line 26))
172.0   Using cached ftfy-6.2.3-py3-none-any.whl.metadata (7.8 kB)
172.4 Collecting tokenizers (from spladerunner==0.1.5->-r requirements.txt (line 28))
172.4   Using cached tokenizers-0.20.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (6.7 kB)
172.7 INFO: pip is looking at multiple versions of spladerunner to determine which version is compatible with other requirements. This could take a while.
172.7 ERROR: Could not find a version that satisfies the requirement onnxruntime-gpu (from spladerunner) (from versions: none)
172.8 ERROR: No matching distribution found for onnxruntime-gpu
------
failed to solve: process "/bin/bash -o pipefail -c python -m pip install --upgrade pip &&     pip install -r requirements.txt &&     python -m spacy download en_core_web_sm" did not complete successfully: exit code: 1

[kosmikdc]

Thanks for the info on this problem. I'll investigate

[lschneidpro]

I ran into the same issue. I removed spladerunner from the requirements, and now it’s working fine. The problem seems to be that one of its dependencies, onnxruntime-gpu, isn’t compatible with Apple Silicon. I’m also curious why spladerunner is included since the book doesn’t mention sparse neural models yet.

[kosmikdc]

Hey guys, thanks isch for that info. I've stubbed splade out to remove the depdency. Now this should be working. I'll wait for further verification before closing, but main should build now.