Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GTiff Reading and Writing #5618

Open
wants to merge 92 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
b6f74e0
replaced custom min/max with std::max/std::min
Kelvinrr Apr 29, 2024
cb12e2e
left a gdal reference in
Kelvinrr Apr 29, 2024
5ccf29a
Initial changes for a GdalIoHandler and creating an external DN cube
acpaquette May 2, 2024
b733739
added write + tests
Kelvinrr May 3, 2024
5d59ed9
Initial move away two flags on the cube to using the LabelAttachment …
acpaquette May 3, 2024
92a4c34
Addressed PR feedback
acpaquette May 8, 2024
9e2e971
Added external gdal support to cube
acpaquette May 8, 2024
2e1cf11
Changed Tiff type to GTiff
acpaquette May 8, 2024
1b72b38
Renamed imageIoHandler to ImageIoHandler
acpaquette May 8, 2024
c4b4754
Added pixeltype to gdaliohandler constructor
acpaquette May 6, 2024
b6c732e
Added gdal pixel type translation and missing special pixels for vari…
acpaquette May 8, 2024
68e8e88
Small fixes to get things working
acpaquette May 8, 2024
d97e25e
Removed prints and added -1s back to read and write
acpaquette May 8, 2024
520efe6
Changed gdal reads to use the raw buffer and correctly apply the ISIS…
acpaquette May 8, 2024
d5fa51a
Added initial pixel conversions and tests for writing
acpaquette May 8, 2024
ab11074
Final tests for pixel type reading and writing
acpaquette May 8, 2024
2d63355
Added tiff fixtures
acpaquette May 9, 2024
a4f3033
Transfered pixel type to gdal handler in cube
acpaquette May 9, 2024
257a0d1
Silenced unsued return warning
acpaquette May 9, 2024
d8da50d
Reused file handler in TiffFixture
acpaquette May 9, 2024
037ef2f
added virtual band checks
Kelvinrr May 10, 2024
1bf4c67
Added default compression for tiffs
acpaquette May 13, 2024
c6c2e32
Removed max again
acpaquette Nov 20, 2024
6d47050
Opened up the environment to get a work env
acpaquette Nov 20, 2024
36bbad0
Fixed boolean comparisons against enum
acpaquette Nov 20, 2024
5152f9b
Initial stab at fixing the buffer copy
acpaquette May 16, 2024
2ba9a48
Adds msk file writting for tiffs
acpaquette May 16, 2024
788059d
Added boundary check for buffers
acpaquette May 16, 2024
b59750a
Added fixture function to write a M x N x O tiff
acpaquette May 23, 2024
8495871
Reverted commented out Index access function Exceptions
acpaquette May 23, 2024
f123f92
Default tiff DNs to the noDataValue
acpaquette May 23, 2024
c67f2db
Cleaned up buffer boundary handling for reads and writes
acpaquette May 23, 2024
9b076bd
Added gdal io tests for reading outside of the image
acpaquette May 23, 2024
e6b4f19
Updated shadow functional tests
acpaquette May 23, 2024
bc185de
Properly handled pointers for mac and linux
acpaquette May 24, 2024
3fd8e72
Removed commented out line
acpaquette May 24, 2024
3cc898c
Addressed OS dependent shadow failures
acpaquette May 25, 2024
9520c3d
Removed read contraints on ecubes
acpaquette Dec 11, 2024
74ba251
Moved CubeIoHandler getters into public space
acpaquette Dec 11, 2024
b221059
Handled gdal warnings in GdalIoHandler and its tests
acpaquette Dec 12, 2024
1e0c056
Fixed compiler warnings in TiffFixture
acpaquette Dec 12, 2024
e95e1b4
Reverted data cube truncation
acpaquette Dec 12, 2024
4c61493
gdal json pvl parsing
Kelvinrr May 15, 2024
11efac8
Update Pvl.cpp
Kelvinrr May 20, 2024
9f89830
Fixed gdal json to pvl parsing
acpaquette Dec 13, 2024
e8484f1
Updated cube tests with tiff read/write changes
acpaquette Dec 13, 2024
315e635
Moved Tiff creation into a function on Cube and made Tiff Cube metada…
acpaquette Dec 16, 2024
ae4e91b
Added PVL to Json export for GDAL metadata
Kelvinrr Jun 24, 2024
ad53b6a
Fixed PVLKeyword toJson writing arrays
acpaquette Dec 30, 2024
064c6ed
Added string conversion for GTiff
acpaquette Dec 30, 2024
62ae4ac
Extracted PVL read json lambda into static function
acpaquette Dec 30, 2024
79a6f61
Enforced Cube carry/manage GdalDataset object and made GdalIoHandler …
acpaquette Dec 30, 2024
bbe2e65
Fixed tiff fixtures alpha bands
acpaquette Dec 30, 2024
98cd446
Seperated ISD spice contructor and init function from default init fu…
acpaquette Jan 6, 2025
f9b0528
Fixed blob reading and writing from hex
acpaquette Jan 6, 2025
cd00d82
Fixed error message when reading a gdal blob
acpaquette Jan 6, 2025
05a23db
Fixed setting dataset ownership correctly on file constructor
acpaquette Jan 6, 2025
d299f87
Fixed closing the dataset if we don't find the USGS metadata
acpaquette Jan 6, 2025
dd27ed1
Fixed cube test expected label size
acpaquette Jan 6, 2025
45ba073
Changed cube blob reading apps to access the blobs via the cube
acpaquette Jan 6, 2025
022d94f
Small fixes
acpaquette Jan 7, 2025
f591bf5
Fixed issue with cam2map requesting buffers completely outside of the…
acpaquette Jan 7, 2025
a0a3b61
Applied cube label reading to catlab
acpaquette Jan 10, 2025
49e46fe
Added blob read exception for GdalRead
acpaquette Jan 10, 2025
8ed18da
Exposed IException vector length
acpaquette Jan 10, 2025
1fed18d
Handled GDALs nodatavalue
acpaquette Jan 10, 2025
5eee306
Moved GDAL pvl label construction into the Cube
acpaquette Jan 10, 2025
245d5ed
Commented out mask creation
acpaquette Jan 10, 2025
2c30951
Forced label reading to come from the cube in ctxcal
acpaquette Jan 10, 2025
eb2a5f3
Fixed data ownership when reading metadata from GTiff
acpaquette Jan 10, 2025
aacf05d
Finalized mapping group generation from spatial reference
acpaquette Jan 17, 2025
44d44f4
Fix reading URLs in qview and blob reading
acpaquette Jan 17, 2025
4d0a84a
Added a scaling factor to the ISIS buffer classes
acpaquette Feb 10, 2025
3b9e8a7
Updated data iohandlers and viewportbuffer to handle scaled buffers
acpaquette Feb 12, 2025
059985c
Finished cube open/create methods to handle GTiffs and Cubes
acpaquette Feb 12, 2025
805799a
Exposed scale on bufferfill action
acpaquette Feb 12, 2025
76b5a50
Updated tests when trying to open a bogus cube
acpaquette Feb 12, 2025
39f4510
Allowed qview file dialog to open cloud volumes
acpaquette Feb 12, 2025
5e7a6e8
Minor buffer indexing fix
acpaquette Feb 12, 2025
9e8508a
Updated GTiff related tests
acpaquette Feb 12, 2025
89cf44b
Added GeoTiff tests
acpaquette Feb 12, 2025
de7c4c8
Converted blob tests to Gtests
acpaquette Feb 12, 2025
47a5c50
Updated Cube to handle detached and external correctly for GTiff format
acpaquette Feb 14, 2025
eacc766
Allow apps to handle other PVL files and fix failing blob tests
acpaquette Feb 18, 2025
efc2871
Updated cube manager tests truth
acpaquette Feb 18, 2025
55fc1ea
Fixed hexstream conversions on linux
acpaquette Feb 19, 2025
d163068
Fix unsigned int comparison in GdalIoHandler
acpaquette Feb 20, 2025
5c63df6
Made TestCubeAttachLineScanTableFromIsd use the temp directory fixture
acpaquette Feb 20, 2025
2290119
Actually fix hex reading and writing
acpaquette Feb 20, 2025
c5e82ce
Set Predictor for GeoTIFFs in ISIS
acpaquette Feb 21, 2025
4c21caa
Removed unused variables causing compiler warnings
acpaquette Feb 21, 2025
29f2949
Set GDAL format from driver
acpaquette Feb 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ dependencies:
- ale =0.10.0,<1
- aom
- armadillo
- boost >=1.78.0,<1.79
- boost-cpp >=1.78.0,<1.79
- boost >=1.76.0,<1.82
- boost-cpp >=1.76.0,<1.82
- blas
- usgs-astrogeology::bullet
- bz2file
Expand All @@ -22,6 +22,7 @@ dependencies:
- ffmpeg
- geos >=3.12, <3.13
- geotiff
- gdal >=3.5 ,<3.9.0a0
- gmp
- graphviz
- conda-forge::gsl >=2.6, <2.7
Expand All @@ -34,10 +35,10 @@ dependencies:
- kalasiris
- krb5
- libgdal >=3.7,<3.8
- libopencv >=4.7,<4.8
- libopencv >=4.5,<4.8
- libpng >=1.6.34,<1.7
- libprotobuf
- libtiff >=4.5,<4.6
- libtiff >=4.5,<4.8
- libxml2
- make
- mesalib
Expand All @@ -47,17 +48,17 @@ dependencies:
- nlohmann_json=3.11.2
- ninja >=1.7.2,<1.8
- nn
- opencv >=4.7,<4.8
- opencv >=4.5,<4.8
- openssl >=3.1.2,<3.2
- pcl >=1.13,<1.14
- plio
- protobuf
- python >=3.10
- python_abi >=3.10
- python >=3.9
- python_abi >=3.9
- pytest
- rclone
- qhull
- qt-main>=5.15.8, <5.16
- qt-main>=5.15, <5.16
- qwt <6.3.0
- sqlite >=3.46.0,<3.47
- suitesparse <7.7.0
Expand Down
2 changes: 1 addition & 1 deletion gtest
Submodule gtest updated 41 files
+1 −1 CMakeLists.txt
+4 −4 CONTRIBUTING.md
+8 −8 README.md
+1 −1 ci/linux-presubmit.sh
+7 −7 docs/advanced.md
+39 −18 docs/faq.md
+0 −6 docs/gmock_cook_book.md
+2 −2 docs/gmock_for_dummies.md
+11 −11 docs/primer.md
+1 −1 docs/reference/assertions.md
+2 −1 docs/reference/mocking.md
+1 −1 docs/reference/testing.md
+10 −10 googlemock/CMakeLists.txt
+3 −3 googlemock/README.md
+1 −7 googlemock/include/gmock/gmock-actions.h
+4 −5 googlemock/include/gmock/gmock-function-mocker.h
+2 −2 googlemock/include/gmock/gmock-matchers.h
+7 −8 googlemock/include/gmock/gmock.h
+1 −1 googlemock/src/gmock-matchers.cc
+0 −9 googlemock/test/gmock-matchers-comparisons_test.cc
+1 −1 googlemock/test/gmock-spec-builders_test.cc
+13 −13 googletest/CMakeLists.txt
+2 −2 googletest/README.md
+19 −19 googletest/cmake/internal_utils.cmake
+10 −9 googletest/include/gtest/gtest-message.h
+7 −29 googletest/include/gtest/gtest-printers.h
+5 −4 googletest/include/gtest/gtest.h
+3 −1 googletest/include/gtest/internal/gtest-death-test-internal.h
+6 −5 googletest/include/gtest/internal/gtest-internal.h
+1 −3 googletest/include/gtest/internal/gtest-param-util.h
+23 −47 googletest/include/gtest/internal/gtest-port.h
+1 −1 googletest/src/gtest-death-test.cc
+1 −1 googletest/src/gtest-filepath.cc
+1 −1 googletest/src/gtest-internal-inl.h
+10 −37 googletest/src/gtest-port.cc
+3 −5 googletest/src/gtest.cc
+3 −3 googletest/test/googletest-port-test.cc
+0 −16 googletest/test/googletest-printers-test.cc
+41 −23 googletest/test/gtest_help_test.py
+1 −1 googletest/test/gtest_unittest.cc
+4 −4 googletest_deps.bzl
3 changes: 2 additions & 1 deletion isis/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,7 @@ find_package(CSM 3.0.3.3 REQUIRED)
find_package(CSPICE 65 REQUIRED)
find_package(Eigen REQUIRED)
find_package(Embree 3.13.0 REQUIRED)
find_package(GDAL REQUIRED CONFIG)
find_package(GeoTIFF 2 REQUIRED)
SET(GSL_ROOT_DIR $ENV{CONDA_PREFIX})
find_package(GSL 2.2.1 REQUIRED)
Expand Down Expand Up @@ -353,7 +354,7 @@ foreach (_variableName ${_variableNames})
endforeach()

# add target based linkages to ALLLIBS variable
list(APPEND ALLLIBS pantor::inja sensorutilities protobuf::libprotobuf embree)
list(APPEND ALLLIBS pantor::inja sensorutilities protobuf::libprotobuf embree GDAL::GDAL)

# Sometimes we add the same lib more than once (especially with LIBDIRS)
list(REMOVE_DUPLICATES ALLLIBDIRS)
Expand Down
6 changes: 3 additions & 3 deletions isis/src/apollo/apps/apollopanstitcher/apollopanstitcher.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -715,11 +715,11 @@ namespace Isis {
att.setFileFormat( panC[0]->format() );
att.setByteOrder( panC[0]->byteOrder() );
att.setPixelType( panC[0]->pixelType() );
if (panC[0]->labelsAttached()) {
att.setLabelAttachment(AttachedLabel);
if (panC[0]->labelsAttached() == Cube::AttachedLabel) {
att.setLabelAttachment(Cube::AttachedLabel);
}
else {
att.setLabelAttachment(DetachedLabel);
att.setLabelAttachment(Cube::DetachedLabel);
}

//define an output cube
Expand Down
14 changes: 11 additions & 3 deletions isis/src/base/apps/cathist/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,17 @@ void IsisMain() {
}

// Extract history from file
Blob historyBlob("IsisCube", "History", fromfile.expanded());
History hist(historyBlob);
Pvl pvl = hist.ReturnHist();
Pvl pvl;
try {
Blob historyBlob("IsisCube", "History", fromfile.expanded());
History hist(historyBlob);
pvl = hist.ReturnHist();
}
catch (...) {
Cube cube(fromfile);
History hist = cube.readHistory();
pvl = hist.ReturnHist();
}

// Print full history
if(mode == "FULL") {
Expand Down
9 changes: 8 additions & 1 deletion isis/src/base/apps/catlab/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,14 @@ void IsisMain() {
QString file = ui.GetCubeName("FROM");

// Extract label from file
Pvl label(file);
Pvl label;
try {
label = Pvl(file);
}
catch (...) {
Cube cube(file);
label = *cube.label();
}

// Output to file if entered
if(ui.WasEntered("TO")) {
Expand Down
5 changes: 3 additions & 2 deletions isis/src/base/apps/catoriglab/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,10 @@ void IsisMain() {
UserInterface &ui = Application::GetUserInterface();
QString file = ui.GetCubeName("FROM");

Pvl fromLabel(file);
Cube fromCube(file);
Pvl fromLabel = *(fromCube.label());
if ( fromLabel.hasObject("OriginalLabel") ) {
OriginalLabel origLab(file);
OriginalLabel origLab = fromCube.readOriginalLabel();
Pvl pvl = origLab.ReturnLabels();
if (ui.IsInteractive()) {
Application::GuiLog(pvl);
Expand Down
2 changes: 1 addition & 1 deletion isis/src/base/apps/crop/crop.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ namespace Isis {
}

// Read the table into a table object
Table table(obj["Name"], from);
Table table = cube->readTable(obj["Name"]);

// We are not going to bother with line/sample associations; they apply
// only to the alpha cube at this time. I'm leaving this code here for the
Expand Down
2 changes: 1 addition & 1 deletion isis/src/base/apps/noproj/noproj.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ namespace Isis {
CubeAttributeOutput cao;

// Can we do a regular label? Didn't work on 12-15-2006
cao.setLabelAttachment(Isis::DetachedLabel);
cao.setLabelAttachment(Cube::DetachedLabel);
FileName matchCubeFile = FileName::createTempFile("$Temporary/match.cub");
QString matchCubeFileNoExt = matchCubeFile.path() + "/" + matchCubeFile.baseName();

Expand Down
2 changes: 1 addition & 1 deletion isis/src/base/apps/shadow/ShadowFunctor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,7 @@ namespace Isis {


Portal portal(3, 3, m_inputDem->pixelType(), -0.5, -0.5);
portal.SetPosition(sample, line, input.Band());
portal.SetPosition(sample + 1, line + 1, input.Band());

if (!portal.CopyOverlapFrom(input)) {
m_inputDem->read(portal);
Expand Down
2 changes: 0 additions & 2 deletions isis/src/base/apps/spiceinit/spiceinit.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,6 @@ namespace Isis {
}
}
icube->deleteGroup("CsmInfo");

p.WriteHistory(*icube);
p.EndProcess();
}
Expand Down Expand Up @@ -592,7 +591,6 @@ namespace Isis {
icube->putGroup(originalKernels);
return false;
}

return true;
}

Expand Down
18 changes: 16 additions & 2 deletions isis/src/base/apps/tabledump/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,14 @@ void IsisMain() {
UserInterface &ui = Application::GetUserInterface();
FileName file = ui.GetCubeName("FROM");
QString tableName = ui.GetString("NAME");
Table table(tableName, file.expanded());
Table table(tableName);
try {
table = Table(tableName, file.expanded());
}
catch (...) {
Cube fromCube(file);
table = fromCube.readTable(tableName);
}

// Set the character to separate the entries
QString delimit;
Expand Down Expand Up @@ -147,7 +154,14 @@ void helperButtonGetTableList() {

UserInterface &ui = Application::GetUserInterface();
QString currentFile = ui.GetCubeName("FROM");
const Pvl label(FileName(currentFile).expanded());
Pvl label;
try {
label = Pvl(FileName(currentFile).expanded());
}
catch (...) {
Cube cube(FileName(currentFile).expanded());
label = *cube.label();
}

// Check to see if the "FILE" parameter has changed since last press
if (currentFile != g_previousFile) {
Expand Down
Loading