-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor implementation of scale/offset, unsigned conversion, and convert missing #1163
refactor implementation of scale/offset, unsigned conversion, and convert missing #1163
Conversation
f62711b
to
36c0790
Compare
I'm currently investigating a performance issue in TDS serving up an NDVI dataset encoded with add_offset and scale_factor. I suspect some sort of memory leak or at best superfluous memory usage. I haven't looked through this PR yet, but do you suspect it might address those issues? Is it in a state where I could try it out myself and see if the problem is better or worse? |
Array data = v.read(); | ||
assertThat((int[]) data.copyTo1DJavaArray()).isEqualTo(expected); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One test I'd be curious to see is if I take values that are from -1 to 1 and encode them as unsigned 8 bit integers (so, scale_factor = 0.008 and add_offset = -1), will the decode process give me back an array of floats rather than doubles. It isn't immediately clear to me that it would do that given that internally the scale and offset are stored as doubles.
There's more changes to follow this PR that should fix some of the performance issues, I'd hold off on testing for a bit. |
36c0790
to
dd11979
Compare
bd4124c
to
3f412b3
Compare
…df-java into scale-offset
72f1815
to
c2c9b74
Compare
c4c7ea1
to
5eabf3a
Compare
cdm/core/src/main/java/ucar/nc2/dataset/EnhanceScaleMissingUnsigned.java
Show resolved
Hide resolved
Co-authored-by: Tara Drwenski <tdrwenski@users.noreply.github.com>
Co-authored-by: Tara Drwenski <tdrwenski@users.noreply.github.com>
Co-authored-by: Tara Drwenski <tdrwenski@users.noreply.github.com>
…df-java into scale-offset
This should be g2g now :) |
No description provided.