Draft
Conversation
pixi run python .\scripts\drive-500x-series.py
Executing notebook.name='5089_set-emissions-history-and-infilling-db.py' with parameters={} from in_notebook=WindowsPath('C:/Users/kikstra/Documents/GitHub/emissions_harmonization_historical/notebooks-papermill/5089_set-emissions-history-and-infilling-db_only_unexecuted.ipynb'). Writing to output_notebook=WindowsPath('C:/Users/kikstra/Documents/GitHub/emissions_harmonization_historical/notebooks-papermill/5089_set-emissions-history-and-infilling-db_only.ipynb')
Executing: 71%|████████████████████████████████████████▋ | 10/14 [00:05<00:02, 1.81cell/s]
Traceback (most recent call last):
File "C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical\scripts\drive-500x-series.py", line 342, in <module>
main()
File "C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical\scripts\drive-500x-series.py", line 201, in main
run_notebook(
File "C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical\scripts\drive-500x-series.py", line 92, in run_notebook
pm.execute_notebook(in_notebook, output_notebook, parameters=parameters)
File "C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical\.pixi\envs\default\Lib\site-packages\papermill\execute.py", line 131, in execute_notebook
raise_for_execution_errors(nb, output_path)
File "C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical\.pixi\envs\default\Lib\site-packages\papermill\execute.py", line 251, in raise_for_execution_errors
raise error
papermill.exceptions.PapermillExecutionError:
---------------------------------------------------------------------------
Exception encountered at "In [6]":
---------------------------------------------------------------------------
PermissionError Traceback (most recent call last)
Cell In[6], line 26
18 with tempfile.NamedTemporaryFile(suffix=".parquet.gzip") as tf:
19 download_zenodo_url(
20 file_info["url"],
21 # We require the interactor while the record's files are embargoed.
(...) 24 size=file_info["size"],
25 )
---> 26 df = pd.read_parquet(tf.name)
28 print(f"Adding {purpose} emissions to the history for harmonisation database")
29 HISTORY_HARMONISATION_DB.save(df.pix.assign(purpose=purpose))
File ~\Documents\GitHub\emissions_harmonization_historical\.pixi\envs\default\Lib\site-packages\pandas\io\parquet.py:667, in read_parquet(path, engine, columns, storage_options, use_nullable_dtypes, dtype_backend, filesystem, filters, **kwargs)
664 use_nullable_dtypes = False
665 check_dtype_backend(dtype_backend)
--> 667 return impl.read(
668 path,
669 columns=columns,
670 filters=filters,
671 storage_options=storage_options,
672 use_nullable_dtypes=use_nullable_dtypes,
673 dtype_backend=dtype_backend,
674 filesystem=filesystem,
675 **kwargs,
676 )
File ~\Documents\GitHub\emissions_harmonization_historical\.pixi\envs\default\Lib\site-packages\pandas\io\parquet.py:267, in PyArrowImpl.read(self, path, columns, filters, use_nullable_dtypes, dtype_backend, storage_options, filesystem, **kwargs)
264 if manager == "array":
265 to_pandas_kwargs["split_blocks"] = True # type: ignore[assignment]
--> 267 path_or_handle, handles, filesystem = _get_path_or_handle(
268 path,
269 filesystem,
270 storage_options=storage_options,
271 mode="rb",
272 )
273 try:
274 pa_table = self.api.parquet.read_table(
275 path_or_handle,
276 columns=columns,
(...) 279 **kwargs,
280 )
File ~\Documents\GitHub\emissions_harmonization_historical\.pixi\envs\default\Lib\site-packages\pandas\io\parquet.py:140, in _get_path_or_handle(path, fs, storage_options, mode, is_dir)
130 handles = None
131 if (
132 not fs
133 and not is_dir
(...) 138 # fsspec resources can also point to directories
139 # this branch is used for example when reading from non-fsspec URLs
--> 140 handles = get_handle(
141 path_or_handle, mode, is_text=False, storage_options=storage_options
142 )
143 fs = None
144 path_or_handle = handles.handle
File ~\Documents\GitHub\emissions_harmonization_historical\.pixi\envs\default\Lib\site-packages\pandas\io\common.py:882, in get_handle(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)
873 handle = open(
874 handle,
875 ioargs.mode,
(...) 878 newline="",
879 )
880 else:
881 # Binary mode
--> 882 handle = open(handle, ioargs.mode)
883 handles.append(handle)
885 # Convert BytesIO or file objects passed with an encoding
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\kikstra\\AppData\\Local\\Temp\\tmpfg0sfn9m.parquet.gzip'
You can still run the infilling without having a filled WMO_2022_PROCESSED_DB, if you have downloaded the data from Zenodo, so let's allow that.
so 4 things to unpack
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cannot run MAGICC on Windows (yet) but trying other core bits.
Also making a few tweaks that may be more personal taste.