Merge pull request #311 from t20100/fcidecomp

Updated Fcidecomp plugin (v2.0.1) and CharLS library (v2.1.0)

doc/information.rst

@@ -77,9 +77,9 @@ HDF5 compression filters and compression libraries sources were obtained from:
   using `c-blosc <https://github.com/Blosc/c-blosc>`_ (v1.21.5), LZ4, Snappy, ZLib and ZStd.
 * hdf5-blosc2 plugin (from `PyTables <https://github.com/PyTables/PyTables/>`_ v3.9.2)
   using `c-blosc2 <https://github.com/Blosc/c-blosc2>`_ (v2.13.2), LZ4, ZLib and ZStd.
-* `FCIDECOMP plugin <ftp://ftp.eumetsat.int/pub/OPS/out/test-data/Test-data-for-External-Users/MTG_FCI_Test-Data/FCI_Decompression_Software_V1.0.2>`_ (v1.0.2)
-  using `CharLS <https://github.com/team-charls/charls>`_
-  (1.x branch, commit `25160a4 <https://github.com/team-charls/charls/tree/25160a42fb62e71e4b0ce081f5cb3f8bb73938b5>`_).
+* `FCIDECOMP plugin <https://gitlab.eumetsat.int/open-source/data-tailor-plugins/fcidecomp>`_
+  (`v2.0.1 <https://gitlab.eumetsat.int/open-source/data-tailor-plugins/fcidecomp/-/tree/e88f83c03bafcd0769c167dca14aa7aabf728e1b>`_)
+  using `CharLS <https://github.com/team-charls/charls>`_ (v2.1.0).
 * `SZ plugin <https://github.com/szcompressor/SZ>`_
   (commit `f466775 <https://github.com/szcompressor/SZ/tree/f4667759ead6a902110e80ff838ccdfddbc8dcd7>`_)
   using `SZ <https://github.com/szcompressor/SZ>`_, ZLib and ZStd.
@@ -116,7 +116,7 @@ Please read the different licenses:
 * blosc2: See `src/PyTables/LICENSE.txt <https://github.com/silx-kit/hdf5plugin/blob/main/src/PyTables/LICENSE.txt>`_  and `src/c-blosc2/LICENSES/ <https://github.com/silx-kit/hdf5plugin/blob/main/src/c-blosc2/LICENSES/>`_
 * bzip2: See `src/PyTables/LICENSE.txt <https://github.com/silx-kit/hdf5plugin/blob/main/src/PyTables/LICENSE.txt>`_ and `src/bzip2/LICENSE <https://github.com/silx-kit/hdf5plugin/blob/main/src/bzip2/LICENSE>`_
 * lz4: See `src/LZ4/COPYING <https://github.com/silx-kit/hdf5plugin/blob/main/src/LZ4/COPYING>`_, `src/LZ4/LICENSE <https://github.com/silx-kit/hdf5plugin/blob/main/src/LZ4/LICENSE>`_ and `src/c-blosc/LICENSES/LZ4.txt <https://github.com/silx-kit/hdf5plugin/blob/main/src/c-blosc/LICENSES/LZ4.txt>`_
-* FCIDECOMP: See `src/fcidecomp/LICENSE <https://github.com/silx-kit/hdf5plugin/blob/main/src/fcidecomp/LICENSE.txt>`_ and `src/charls/src/License.txt  <https://github.com/silx-kit/hdf5plugin/blob/main/src/charls/src/License.txt>`_
+* FCIDECOMP: See `src/fcidecomp/LICENSE <https://github.com/silx-kit/hdf5plugin/blob/main/src/fcidecomp/LICENSE>`_ and `src/charls/LICENSE.md  <https://github.com/silx-kit/hdf5plugin/blob/main/src/charls/LICENSE.md>`_
 * SZ: See `src/SZ/copyright-and-BSD-license.txt <https://github.com/silx-kit/hdf5plugin/blob/main/src/SZ/copyright-and-BSD-license.txt>`_
 * SZ3: See `src/SZ3/copyright-and-BSD-license.txt <https://github.com/silx-kit/hdf5plugin/blob/main/src/SZ3/copyright-and-BSD-license.txt>`_
 * zfp: See `src/H5Z-ZFP/LICENSE <https://github.com/silx-kit/hdf5plugin/blob/main/src/H5Z-ZFP/LICENSE>`_ and `src/zfp/LICENSE <https://github.com/silx-kit/hdf5plugin/blob/main/src/zfp/LICENSE>`_

setup.py

@@ -467,6 +467,7 @@ def finalize_options(self):
             self.distribution.ext_modules = [
                 ext for ext in self.distribution.ext_modules
                 if '-std=c++14' not in ext.extra_compile_args
+                and not (isinstance(ext, HDF5PluginExtension) and ext.cpp14_required)
             ]
 
         if not self.hdf5plugin_config.use_cpp20:
@@ -605,7 +606,17 @@ def build_extensions(self):
 class HDF5PluginExtension(Extension):
     """Extension adding specific things to build a HDF5 plugin"""
 
-    def __init__(self, name, sse2=None, avx2=None, cpp11=None, cpp11_required=False, cpp20_required=False, **kwargs):
+    def __init__(
+            self,
+            name,
+            sse2=None,
+            avx2=None,
+            cpp11=None,
+            cpp11_required=False,
+            cpp14_required=False,
+            cpp20_required=False,
+            **kwargs
+    ):
         Extension.__init__(self, name, **kwargs)
 
         if not self.depends:
@@ -632,6 +643,7 @@ def __init__(self, name, sse2=None, avx2=None, cpp11=None, cpp11_required=False,
         self.avx2 = avx2 if avx2 is not None else {}
         self.cpp11 = cpp11 if cpp11 is not None else {}
         self.cpp11_required = cpp11_required
+        self.cpp14_required = cpp14_required
         self.cpp20_required = cpp20_required
 
     @property
@@ -664,8 +676,9 @@ def get_charls_clib(field=None):
 
     config = dict(
         sources=glob(f'{charls_dir}/*.cpp'),
-        include_dirs=[charls_dir],
-        cflags=['-std=c++11'],
+        include_dirs=["src/charls/include", "src/charls/include/charls", charls_dir],
+        macros=[("CHARLS_STATIC", 1), ("CHARLS_LIBRARY_BUILD", 1)],
+        cflags=['-std=c++14'],
     )
 
     if field is None:
@@ -1100,21 +1113,20 @@ def get_bzip2_plugin():
 
 def get_fcidecomp_plugin():
     """FCIDECOMP plugin build config"""
-    fcidecomp_dir = 'src/fcidecomp/FCIDECOMP_V1.0.2/Software/FCIDECOMP_SOURCES'
+    fcidecomp_dir = 'src/fcidecomp/src/fcidecomp'
 
     extra_compile_args = ['-O3', '-ffast-math', '-std=c99', '-fopenmp']
     extra_compile_args += ['/Ox', '/fp:fast', '/openmp']
-    extra_link_args = ['-fopenmp', '/openmp']
+    extra_link_args = ['-lstdc++', '-fopenmp', '/openmp']
 
     return HDF5PluginExtension(
         "hdf5plugin.plugins.libh5fcidecomp",
         sources=glob(f"{fcidecomp_dir}/fcicomp-*/src/*.c"),
         include_dirs=glob(f"{fcidecomp_dir}/fcicomp-*/include") + get_charls_clib('include_dirs'),
         extra_compile_args=extra_compile_args,
         extra_link_args=extra_link_args,
-        cpp11={'extra_link_args': ['-lstdc++']},
-        cpp11_required=True,
-        define_macros=[('CHARLS_STATIC', 1)],
+        cpp14_required=True,
+        define_macros=[('CHARLS_STATIC', 1), ('LOGGING', 1)],
     )
 
 

src/charls/.clang-format

@@ -0,0 +1,41 @@
+# Copyright (c) Team CharLS.
+# SPDX-License-Identifier: BSD-3-Clause
+
+BasedOnStyle: LLVM
+Language: Cpp
+
+# Use features of C++11, C++14 and C++17 (e.g. A<A<int>> instead of A<A<int> >)
+Standard: Cpp11
+
+UseTab: Never
+IndentWidth: 4
+ColumnLimit: 0
+PointerAlignment: Left
+
+BreakBeforeBraces: Custom
+BraceWrapping:
+  AfterClass: true
+  AfterControlStatement: true
+  AfterEnum: true
+  AfterFunction: true
+  AfterNamespace: false
+  AfterObjCDeclaration: true
+  AfterStruct: true
+  AfterUnion: true
+  BeforeCatch: true
+  BeforeElse: true
+  IndentBraces: false
+
+AccessModifierOffset: -4
+AlignTrailingComments: true
+AllowShortFunctionsOnASingleLine: true
+AllowShortIfStatementsOnASingleLine: true
+AlwaysBreakTemplateDeclarations: true
+BreakConstructorInitializers: AfterColon
+CompactNamespaces: true
+ConstructorInitializerAllOnOneLineOrOnePerLine: true
+SpaceAfterTemplateKeyword: false
+SpaceBeforeCpp11BracedList: false
+KeepEmptyLinesAtTheStartOfBlocks: false
+MaxEmptyLinesToKeep: 2
+NamespaceIndentation: None

src/charls/.editorconfig

@@ -1,8 +1,57 @@
+# Copyright (c) Team CharLS.
+# SPDX-License-Identifier: BSD-3-Clause
+
 root = true
 
-[{src}/**.{h,cpp}]
+# Default settings
+[*]
 charset = utf-8
+indent_style = space
 trim_trailing_whitespace = true
+
+[*.{h,cpp}]
+indent_size = 4
 insert_final_newline = true
-indent_style = space
-indent_size = 4
+
+[CMakeLists.txt]
+indent_size = 2
+
+[*.cs]
+indent_size = 4
+
+# New line preferences
+csharp_new_line_before_catch = true
+csharp_new_line_before_else = true
+csharp_new_line_before_open_brace = control_blocks, types, methods, object_collection, accessors, properties
+
+# Indentation preferences
+csharp_indent_case_contents = true
+csharp_indent_switch_labels = true
+
+# Space preferences
+csharp_space_after_cast = false
+csharp_space_after_colon_in_inheritance_clause = true
+csharp_space_after_keywords_in_control_flow_statements = true
+csharp_space_before_colon_in_inheritance_clause = true
+csharp_space_between_method_call_empty_parameter_list_parentheses = false
+csharp_space_between_method_call_name_and_opening_parenthesis = false
+csharp_space_between_method_call_parameter_list_parentheses = false
+csharp_space_between_method_declaration_empty_parameter_list_parentheses = false
+csharp_space_between_method_declaration_parameter_list_parentheses = false
+
+csharp_preserve_single_line_blocks = true
+
+csharp_style_expression_bodied_constructors = false:suggestion
+csharp_style_expression_bodied_methods = false:suggestion
+csharp_style_expression_bodied_properties = true:suggestion
+csharp_style_inlined_variable_declaration = false:suggestion
+csharp_style_var_for_built_in_types = true:suggestion
+csharp_style_var_when_type_is_apparent = true:suggestion
+
+dotnet_sort_system_directives_first = true
+
+dotnet_style_predefined_type_for_member_access = true:suggestion
+dotnet_style_predefined_type_for_locals_parameters_members = true:suggestion
+dotnet_style_qualification_for_field = false:suggestion
+dotnet_style_qualification_for_method = false:suggestion
+dotnet_style_qualification_for_property = false:suggestion

src/charls/.gitattributes

@@ -5,6 +5,7 @@
 *.c text diff=cpp
 *.cpp text diff=cpp
 *.h text diff=cpp
+*.cs text diff=csharp
 
 # Explicit exclude binary files from the line ending normalization.
 *.png binary

src/charls/.gitignore

@@ -6,17 +6,29 @@ Win32/
 x64/
 .vscode/
 .vs/
+build/
 [Dd]ebug/
 [Rr]elease/
 [Cc]hecked/
-build/
+TestResults/
+*.opensdf
+*.sdf
 
 # Ignore files created by Visual Studio
 *.suo
 *.user
 *.ipch
-*.opensdf
-*.sdf
 *.opendb
-*.VC.db
-*.aps
+*.db
+*.aps
+*.sqlite
+
+# NuGet Packages
+*.nupkg
+# The packages folder can be ignored because of Package Restore
+**/packages/*
+# except build/, which is used as an MSBuild target.
+!**/packages/build/
+
+# Ignore Coverity build folder
+cov-int/

src/charls/.travis.yml

@@ -1,28 +1,22 @@
+dist: bionic
+
 language: cpp
 
 compiler:
   - gcc
   - clang
 
-install:
-- if [ "$CXX" = "g++" ]; then export CXX="g++-4.8" CC="gcc-4.8"; fi
-- if [ "$CXX" = "clang++" ]; then export CXX="clang++-3.7" CC="clang-3.7"; fi
-
 addons:
   apt:
     sources:
     - ubuntu-toolchain-r-test
     - llvm-toolchain-precise-3.7
-    packages:
-    - gcc-4.8
-    - g++-4.8
-    - clang-3.7
 
 sudo: false
 
 before_script:
   - mkdir debug
   - cd debug
-  - cmake -DCMAKE_BUILD_TYPE=Debug ..
+  - cmake -DCMAKE_BUILD_TYPE=Debug -DCHARLS_PEDANTIC_WARNINGS=On -DCHARLS_THREAT_WARNINGS_AS_ERRORS=On ..
 
-script: make
+script: make

src/charls/CHANGELOG.md

@@ -0,0 +1,69 @@
+# Change Log
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/).
+
+## [2.1.0] - 2019-12-29
+
+### Added
+
+- Two new C++ classes (jpegls_encoder \ jpegls_decoder) have been added to make it much easier to use CharLS from C++
+- A new C API (charls_xxx functions) was added to provide a more stable ABI for future updates. The old API calls are internally forwarded to the new API.
+- CharLS can now read and write JPEG-LS standard SPIFF headers
+- Support has been added to detect the unsupported JPEG-LS extension (ISO/IEC 14495-2) SOF_57 marker and IDs in LSE marker
+- The unit test project has been extended and now includes 188 tests.
+- Support has been added to encode\decode 4 component images in all interleave modes
+
+### Deprecated
+
+- The legacy 1.x\2.0 C API has been marked as deprecated. This legacy API will be maintained until the next major upgrade.
+  Future 2.x updates will start to mark the legacy types and functions with the C++ ```[[deprecated]]``` attribute.
+
+### Changed
+
+- charls_error has been replaced by a C++11 compatible jpegls_errc error code enum design
+- The included C and C++ sample have been updated to use the new C\C++ API
+- Improved the validation of the JPEG-LS stream during decoding
+- #pragma once is now used to prevent that header files are included multiple times (supported by all modern C++ compilers)
+- The referenced NuGet packages of the .NET wrapper assembly are updated to their latest versions
+- The CMake build script has been updated to modern CMake and requires at least CMake 3.9
+- All types are now in the charls C++ namespace
+- All source code files now use the SPDX Unique License Identifiers (BSD-3-Clause) to identify the license
+
+### Removed
+
+- Support to write JFIF headers during encoding has been removed. JFIF headers were already skipped when present during decoding.
+  SPIFF headers should be used when creating standalone .jls files
+- Support for .NET Code Contracts has been removed as this technology is being phased out by Microsoft
+
+### Fixed
+
+- Fixed [#7](https://github.com/team-charls/charls/issues/7), How to compile CharLS with Xcode has been documented in the Wiki
+- Fixed [#44](https://github.com/team-charls/charls/issues/44), Only the API functions should be exported from a Linux shared library
+- Fixes [#35](https://github.com/team-charls/charls/issues/35), Encoding will fail if the bit per sample is greater than 8, and a custom RESET value is used
+- Fixes [#51](https://github.com/team-charls/charls/issues/51), The default threshold values are not corrected computed for 6 bit images or less
+- Fixed the ASSERT in the ModuloRange function, which would trigger false assertions in debug builds
+
+## [2.0.0] - 2016-5-18
+
+### Changed
+
+- Updated the source code to C++14
+- Refactored several APIs to make usage of the library easier
+
+### Fixed
+
+- Fixes [#10](https://github.com/team-charls/charls/issues/10), Fixed the problem that "output buffer to small" was not detected when writing encoded bytes to a fixed output buffer. This could cause memory corruption problems
+- Fixes [11](https://github.com/team-charls/charls/issues/11), Update charlstest to return EXIT_SUCCESS/FAILURE
+- Fixed the issue that DecodeToPnm would set params.colorTransform = ColorTransformation::BigEndian but the library didn’t support this option during decoding
+
+## [1.1.0] - 2016-5-15
+
+### Fixed
+
+- Fixes [#9](https://github.com/team-charls/charls/issues/9) EncoderStrategy::AppendToBitStream method fails if buffer is full, 31 bits are added and xFF bytes are written
+
+## [1.0.0] - 2010-11-18
+
+First release of the CharLS JPEG-LS codec.