Incosistent data types and OverflowError

[konstantinos2018]

Relevant commit

Commit: a7e40d0

Dependencies

OS: Ubuntu 24.04
Python version: Python 3.13.1
Conda version: conda 24.9.2
mamba version: 2.0.5

Issue Description

I've encountered the following two Overflow errors and made some workarounds which made polymer to run properly, although I haven't tested whether the atmospherically corrected products are as they should be:

Error 1

Traceback (most recent call last):
  File "/home/kostageo/Projects/polymer/my_example.py", line 14, in <module>
    run_atm_corr(input_obj, output_obj, multiprocessing=0)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kostageo/Projects/polymer/polymer/main.py", line 522, in run_atm_corr
    for block in block_iter:
                 ^^^^^^^^^^
  File "/home/kostageo/Projects/polymer/polymer/main.py", line 438, in blockiterator
    for block in level1.blocks(params.bands_read()):
                 ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/home/kostageo/Projects/polymer/polymer/level1_msi.py", line 496, in blocks
    yield self.read_block(size, offset, bands_read)
          ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kostageo/Projects/polymer/polymer/level1_msi.py", line 397, in read_block
    Rtoa = (raw_data + self.radio_offset_list[iband])/self.quantif
            ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OverflowError: Python integer -1000 out of bounds for uint16

Workaround

Change "/home/kostageo/Projects/polymer/polymer/level1_msi.py", line 397 to
Rtoa = (raw_data.astype('float32') + self.radio_offset_list[iband])/self.quantif

Error 2

Traceback (most recent call last):
  File "/home/kostageo/Projects/polymer/my_example.py", line 14, in <module>
    run_atm_corr(input_obj, output_obj, multiprocessing=0)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kostageo/Projects/polymer/polymer/main.py", line 524, in run_atm_corr
    l2.write(block)
    ~~~~~~~~^^^^^^^
  File "/home/kostageo/Projects/polymer/polymer/level2.py", line 122, in write
    self.write_block(d, block[d], S,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
                     block.attributes.get(d, {}))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kostageo/Projects/polymer/polymer/level2_nc.py", line 117, in write_block
    data[np.isnan(data)] = fill_value
    ~~~~^^^^^^^^^^^^^^^^
OverflowError: Python integer -32767 out of bounds for uint16

Workaround

Add before "/home/kostageo/Projects/polymer/polymer/level2_nc.py", line 117 the following
data = data.astype('float32')

[fsteinmetz]

Indeed, this seems to be a remaining incompatibility with numpy2.
As a workaround, you can also install an env with numpy<2.
I will fix this, thanks for the feedback.