-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add AWSS3 as a package extension #147
Conversation
Hm, codecov says the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is great! another thing we've talked about internally in teh context of using a package extension here is supporting atomically capturing the versionId generated on write. we can't do that without having methods for Onda.store(::S3Path, ...)
I think.
that might be more complex than we want to tackle in this initial PR though so I'm happy to let it go for now!
test/awss3.jl
Outdated
# Load subspan to exercise method | ||
span = TimeSpan(0, Second(1)) | ||
loaded_span = Onda.load(signal, span; encoded=true) | ||
@test loaded_samples[:, span] == loaded_span |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for some reason it doesn't seem like this has hit that added method, at least according to the "missed coverage" annotations above. I wonder if somehow the package extension isn't getting loaded properly in tests?
we also have some tests in OndaBatches.jl which try to really make sure we're actually passing the byte range to AWSS3: https://github.com/beacon-biosignals/OndaBatches.jl/blob/a43b6f0a6837c2a478a1845c69728c1851e234c0/test/utils.jl#L22-L27
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
locally I confirmed it was loading, but I agree something seems off here. Weirdly codecov says the other method WAS hit in the extension code...
Co-authored-by: Dave Kleinschmidt <dave.f.kleinschmidt@gmail.com>
Turns out the issue was sloppily copying the code to create the signal, since I was using compressed LPCM which of course we don't do byte range requests for currently |
Remaining coverage loss is we aren't hitting the |
hmm, I don't off teh top of my head. but it does seem to get hit in OndaBatches.jl tests; https://app.codecov.io/gh/beacon-biosignals/OndaBatches.jl/blob/main/src%2Futils.jl#L18 |
I think that path gets hit when you load without a span... |
We already do try that, but I suspect maybe that happens in the compressed case, so going to try both compressed & uncompressed like in the OndaBatches tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
few little formatting thing (no auto-formatter apparently) but
Co-authored-by: Dave Kleinschmidt <dave.f.kleinschmidt@gmail.com>
To automatically use the efficient byte-range method when AWSS3 is loaded, on Julia 1.9+. This is currently pirated in various places, which should be removed on 1.9+.
I have used the strategy from here such that the feature is only present on 1.9+, but there's no dependency cost pre-1.9. An alternative would be to add AWSS3 as a full dependency pre-1.9.