Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: ungarj/mapchete
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 2024.12.0
Choose a base ref
...
head repository: ungarj/mapchete
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
  • 11 commits
  • 9 files changed
  • 3 contributors

Commits on Jan 9, 2025

  1. limit matplotlib below version 3.10

    ungarj committed Jan 9, 2025
    Copy the full SHA
    080d434 View commit details
  2. don't install python-tk

    ungarj committed Jan 9, 2025
    Copy the full SHA
    f7b14e5 View commit details
  3. Merge pull request #662 from ungarj/fix_pipeline_202501

    fix pipeline
    ungarj authored Jan 9, 2025
    6

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    2edc31c View commit details
  4. some more flags and asserts to test_resample_from_array, make masked_…

    …arrays nd.arrays for rasterio>1.4.0 reproject
    sevcikp committed Jan 9, 2025
    Copy the full SHA
    bcba03d View commit details
  5. run pre-commit

    sevcikp committed Jan 9, 2025
    Copy the full SHA
    c679a45 View commit details
  6. also add mask as invertion of the test array for mask sake for the te…

    …st_resample_from_array
    sevcikp committed Jan 9, 2025
    Copy the full SHA
    69a9a70 View commit details

Commits on Jan 10, 2025

  1. Merge pull request #663 from ungarj/rasterio_reproject_no_in_masked_a…

    …rray
    
    rasterio>=1.4.x reproject array type
    ungarj authored Jan 10, 2025
    6

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    ea96090 View commit details

Commits on Jan 14, 2025

  1. Release 2025.1.0

    ungarj committed Jan 14, 2025
    6
    Copy the full SHA
    d01bef0 View commit details

Commits on Jan 27, 2025

  1. properly handle region_name in MPath for S3 paths

    ungarj committed Jan 27, 2025
    Copy the full SHA
    8a95315 View commit details
  2. Merge pull request #667 from ungarj/fix666_cli_handle_devilish_s3fs_r…

    …egion_name
    
    properly handle region_name in MPath for S3 paths
    ungarj authored Jan 27, 2025
    6

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    c22dcc6 View commit details
  3. Release 2025.1.1

    ungarj committed Jan 27, 2025
    6
    Copy the full SHA
    2ba3189 View commit details
Showing with 66 additions and 6 deletions.
  1. +1 −1 .github/workflows/python-package.yml
  2. +23 −1 CHANGELOG.rst
  3. +1 −1 LICENSE
  4. +1 −1 mapchete/__init__.py
  5. +3 −0 mapchete/io/raster/array.py
  6. +7 −0 mapchete/path.py
  7. +2 −2 pyproject.toml
  8. +20 −0 test/test_io_raster.py
  9. +8 −0 test/test_path.py
2 changes: 1 addition & 1 deletion .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
@@ -40,7 +40,7 @@ jobs:
run: |
sudo apt-add-repository -y ppa:ubuntugis/ubuntugis-unstable
sudo apt-get -y update
sudo apt-get install -y gdal-bin python-tk libgdal-dev libproj-dev libgeos-dev
sudo apt-get install -y gdal-bin libgdal-dev libproj-dev libgeos-dev
python -m pip install --upgrade pip wheel
pip install -r test/requirements.txt
pip install -e .[complete]
24 changes: 23 additions & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
@@ -3,8 +3,30 @@ Changelog
#########


---------------------
2025.1.1 - 2025-01-27
---------------------

* core

* `path.MPath`: properly handle `region_name` for S3 paths #667


---------------------
2025.1.0 - 2025-01-14
---------------------

* core

* `io.raster.array.resample_from_array()`: fix regression caused by new behavior of `rasterio.warp.reproject()` (#663)

* packaging

* limit `matplotlib` below version `3.10`


----------------------
2024.11.1 - 2024-11-25
2024.12.0 - 2024-12-03
----------------------

* core
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The MIT License (MIT)

Copyright (c) 2015 - 2024 EOX IT Services
Copyright (c) 2015 - 2025 EOX IT Services

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
2 changes: 1 addition & 1 deletion mapchete/__init__.py
Original file line number Diff line number Diff line change
@@ -38,7 +38,7 @@
"VectorInputGroup",
"ZoomLevels",
]
__version__ = "2024.12.0"
__version__ = "2025.1.1"

logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler())
3 changes: 3 additions & 0 deletions mapchete/io/raster/array.py
Original file line number Diff line number Diff line change
@@ -169,6 +169,9 @@ def resample_from_array(

dst_data = np.empty(dst_shape, array.dtype)

if isinstance(array, ma.MaskedArray):
array = array.data

reproject(
array,
dst_data,
7 changes: 7 additions & 0 deletions mapchete/path.py
Original file line number Diff line number Diff line change
@@ -158,6 +158,13 @@ def fs(self) -> fsspec.AbstractFileSystem:
if self._fs is not None:
return self._fs
elif self._path_str.startswith("s3://"):
# move 'region_name' up to client_kwargs in order to have effect
if self._storage_options.get("region_name"):
client_kwargs = self._storage_options.get("client_kwargs", {})
client_kwargs.update(
region_name=self._storage_options.pop("region_name")
)
self._storage_options.update(client_kwargs=client_kwargs)
return fsspec.filesystem(
"s3",
requester_pays=self._storage_options.get(
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -57,7 +57,7 @@ complete = [
"Flask-RangeRequest",
"fsspec[http,s3]>=2023.12.0",
"lxml",
"matplotlib",
"matplotlib<3.10",
"memray",
"pystac[urllib3]>=1.8.2",
"requests",
@@ -67,7 +67,7 @@ complete = [
"werkzeug>=0.15",
]
contours = [
"matplotlib",
"matplotlib<3.10",
]
http = [
"aiohttp",
20 changes: 20 additions & 0 deletions test/test_io_raster.py
Original file line number Diff line number Diff line change
@@ -401,6 +401,26 @@ def test_resample_from_array():
out_array = resample_from_array(in_data, in_tile.affine, out_tile)
# deprecated
resample_from_array(in_data, in_tile.affine, out_tile, nodata=-9999)
# keep 2D
in_data = (np.ones(in_tile.shape[1:]),)
out_tile = BufferedTilePyramid("geodetic").tile(6, 10, 10)
out_array = resample_from_array(in_data, in_tile.affine, out_tile, keep_2d=True)
assert out_array.shape == (256, 256)
# not keep 2D
in_data = (np.ones(in_tile.shape[1:]),)
out_tile = BufferedTilePyramid("geodetic").tile(6, 10, 10)
out_array = resample_from_array(in_data, in_tile.affine, out_tile, keep_2d=False)
assert out_array.shape == (1, 256, 256)
# Test ma.MaskedArray input
in_data = (
ma.MaskedArray(
data=np.ones(in_tile.shape[1:]),
mask=np.invert(np.ones(in_tile.shape[1:]), dtype="bool", casting="unsafe"),
),
)
out_tile = BufferedTilePyramid("geodetic").tile(6, 10, 10)
out_array = resample_from_array(in_data, in_tile.affine, out_tile)
assert out_array.shape == (1, 256, 256)
# errors
with pytest.raises(TypeError):
in_data = "invalid_type"
8 changes: 8 additions & 0 deletions test/test_path.py
Original file line number Diff line number Diff line change
@@ -527,3 +527,11 @@ def test_with_protocol_http():
def test_with_protocol_relative():
path = MPath("foo/bar")
assert path.with_protocol("https") == "https://foo/bar"


def test_s3_region_name():
path = MPath("s3://foo", storage_options=dict(region_name="bar"))
# trigger a rewrite of storage options to fit with S3FS
path.fs
# not an ideal way to test this, but the storage_options are passed on to S3FS that way
assert path._storage_options.get("client_kwargs", {}).get("region_name") == "bar"