Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCS support #120

Open
kentslaney opened this issue Oct 6, 2024 · 1 comment
Open

GCS support #120

kentslaney opened this issue Oct 6, 2024 · 1 comment

Comments

@kentslaney
Copy link

While trying to load a TFDS dataset from GCS, I encountered this error:

/usr/local/lib/python3.10/dist-packages/array_record/python/array_record_data_source.py in _create_reader
    176 def _create_reader(filename: epath.PathLike):
    177   """Returns an ArrayRecordReader for the given filename."""
--> 178   return array_record_module.ArrayRecordReader(
    179       filename,
    180       options="readahead_buffer_size:0",

RuntimeError: open() failed: No such file or directory; opening gs://ref_coco/ref_coco/refcoco_unc/1.1.0/ref_coco-train.array_record-00000-of-00032

See the minimum reproducible example here:

https://colab.research.google.com/drive/1iezVDZBJrWtP3fVTpAMNGb6qfENZNKuD?usp=sharing

@kentslaney
Copy link
Author

I should mention that I'm not blocked right now because of this, the dataset is small enough to fit on disk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant