Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adsk Contrib - Issues #1968 (mirrored) and #1992 (LUT-free builtins) #2029

Conversation

cozdas
Copy link
Collaborator

@cozdas cozdas commented Aug 27, 2024

For issue #1992 - "Make LUT-free implementations of certain built-in transforms"

  • Adding fixed-function FIXED_FUNCTION_LIN_TO_PQ. This will be used to handle following built-in transforms if the LUT support is turned off:
    -- CURVE - ST-2084_to_LINEAR
    -- CURVE - LINEAR_to_ST-2084
    -- DISPLAY - CIE-XYZ-D65_to_REC.2100-PQ
    -- DISPLAY - CIE-XYZ-D65_to_ST2084-P3-D65

  • Adding fixed function FIXED_FUNCTION_LIN_TO_GAMMA_LOG. This is a parametrized gamma + log segment curve with adjustable mirroring point. This will be used to handle following built-in transform if the LUT support is turned off:
    --CURVE - HLG-OETF
    --CURVE - HLG-OETF-INVERSE
    -- DISPLAY - CIE-XYZ-D65_to_REC.2100-HLG-1000nit
    -- CURVE - APPLE_LOG_to_LINEAR
    -- APPLE_LOG_to_ACES2065-1

  • Adding fixed function FIXED_FUNCTION_LIN_TO_DOUBLE_LOG which has two log-affine segments with an optional linear segment in-between. This will be used to handle following built-in transform if the LUT support is turned off:
    -- CANON_CLOG2-CGAMUT_to_ACES2065-1
    -- CURVE - CANON_CLOG2_to_LINEAR
    -- CANON_CLOG3-CGAMUT_to_ACES2065-1
    -- CURVE - CANON_CLOG3_to_LINEAR

  • Implemented CPU renderers for scalar, SSE2 (with fastPower) and SSE2 (with Intel SVML) intrinsics targets for the new PQ fixed-function. The other two fixed-functions are implemented in the basic form and will need more optimization in the future when we introduce the option to turn the LUT support off.

  • Implemented GPU shader generator for all the new fixed-functions,

For issue #1968 - "Make display EOTFs in built-in transforms mirrored and unclamped to preserve sub-black and super-white"

  • Both the existing LUT-based and the new LUT-free implementations using PQ or HLG curves are now unclamped at both ends and mirrored around the origin.
  • Added new built-in transforms:
    --CURVE - HLG-OETF
    --CURVE - HLG-OETF-INVERSE

Aux Changes

  • GetFixedFunctionCPURenderer() now takes a new bool parameter fastLogExpPow.
  • Added a stand-in OCIO_LUT_SUPPORT preprocessor macro in preparation for ocio-lite where the lut support can be turned off.
  • Added util functions to Config_tests.cpp to help creating configs with arbitrary version.
  • Added ability to test fixed-function cpu renderers with and without fastLogExpPow.
  • Added capability to specify custom extended ranges in the GPU unit tests. Default value is [-1.0,2.0] same as the previously hard-coded range.
  • Fixed a bug in the GPU unit tests where the computed domain values would overshoot.
  • In BuiltinTransform_tests it's now possible (and necessary) to specify the error threshold for each built-in transform. This was needed as the hard-coded 1e-6 threshold was too tight for the PQ curve which uses nested power functions and thus numerical stability was depending on the implementation.
  • Added GpuShaderText::floatXGreaterThanEqual() functions

…ns) and AcademySoftwareFoundation#1992 (LUT-free builtins Pt.1)

For issue AcademySoftwareFoundation#1992 - "Make LUT-free implementations of certain built-in transforms"
- Adding fixed-functions PQ_TO_LINEAR and LINEAR_TO_PQ. These will be used to handle following built-in transforms if the LUT support is turned off:
-- CURVE - ST-2084_to_LINEAR
-- CURVE - LINEAR_to_ST-2084
-- DISPLAY - CIE-XYZ-D65_to_REC.2100-PQ
-- DISPLAY - CIE-XYZ-D65_to_ST2084-P3-D65
- Implemented CPU renderers for scalar, SSE2 (with fastPower) and SSE2 (with Intel SVML) intrinsics targets for the new fixed-function.
- Implemented GPU shader generator for the new fixed-function
- Remaining fixed-functions will be implemented in an upcoming PR.

For issue AcademySoftwareFoundation#1968 - "Make display EOTFs in built-in transforms mirrored and unclamped to preserve sub-black and super-white"
- Existing LUT-based HLG curve is now unclamped at both ends and mirrored around origin.
- Both the existing LUT-based and the new LUT-free implementations of the PQ curve are now unclamped at both ends and mirrored around origin.

Aux Changes
- GetFixedFunctionCPURenderer() now takes a new bool parameter fastLogExpPow.
- Added a stand-in OCIO_LUT_SUPPORT preprocessor macro in preparation for ocio-lite where the lut support can be turned off.
- Added util functions to Config_tests.cpp to help creating configs with arbitrary version.
- Added ability to test fixed-function cpu renderers with and without fastLogExpPow.
- Added capability to specify custom extended ranges in the GPU unit tests. Default value is [-1.0,2.0] same as the previously hard-coded range.
- Fixed a bug in the GPU unit tests where the computed domain values would overshoot.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
@cozdas cozdas force-pushed the ozdasc/main/BuiltIns_LutFreePt1_Mirrored branch from 350165f to 2ce223f Compare August 27, 2024 23:07
…ted to MSVC 2019+ compiler as it has the built-in intel _mm_pow_ps() SVML implementation.

- LUT-based PQ to Linear implementation now uses half-domain LUT to handle extended range. Linear to PQ direction was alredy using half-domain lut (probably due to normal input range being 0 to 100.
- BT_2100 namespace is renamed to HLG
- In BuiltinTransform_tests it's now possible (and necessary) to specify the error threshold for each built-in transform. This was needed as the hard-coded 1e-6 threshold was too tight for the PQ curve which uses nested power functions and thus numerical stability was depending on the implantation flavor.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
@@ -365,128 +363,190 @@ AllValues UnitTestValues
{
// Contains the name, the input values and the expected output values.

{ "IDENTITY",
{ { 0.5f, 0.4f, 0.3f }, { 0.5f, 0.4f, 0.3f } } },
{ "IDENTITY",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding ability to specify different thresholds for different built-ins as some are more sensitive than others to the underlying implementation change (i.e. lut-based vs scalar vs SSE implementation). Increased the threshold for PQ curve which has nested power() invocation thus numerically less stable. Also adding extended range values to make sure that all implementations can handle wider than [0,1] domain. Target values are computed with double-precision math.

@cozdas cozdas marked this pull request as draft September 5, 2024 00:01
…rection is linear -> PQ . Changed the enum names accordinly, updated the tests etc.

- Adding a new Hybrid Log Gamma fixed function. Similar to the PQ curve, the forward direction is linear -> HLG. SIMD implementation is currently missing, can be done in the future.
- minor fixes in the PQ cpu op
- some adjustments to the CPU test values and thresholds.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
… again.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
- Code beautification.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
- Since we changed the default direction of the PQ and HLG fixed functions, I moved the order of the functions in the source files so that forward becomes before inverse.
- Adding the first cut of the FIXED_FUNCTION_LINEAR_TO_DOUBLE_LOG_AFFINE fixed function. This is still in WIP, needs some optimizations and tests implemented. Also to be able to handle the Apple Camera curve, it needs to be extended a bit. But currently it can handle the Canon CLog2 and CLog3 curves with all tests passing.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
…needed for precise break-point handling in non-continuous piecewise functions.

- changes for double log gamma fixed function
-- Fixed couple of bugs
-- Added basic parameter validation
-- implemented fwd and inv gpu shaders
-- Fixed CLog2 and CLog3 double log gamma constants.
-- implemented cpu and gpu shaders

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
…d Rec.2100 HLG curve parameters are now passed to the fixed-function by the built-in transforms and by some of the test suites.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
…nt mirroring is applied. Below the mirror point the input values will be mirrored around that point and the output sign will be flipped. Although this does not guarantee by itself a monotonously increasing (thus invertible) and a continuous function, when the parameters are proper these properties will be achieved.

- Both the Rec.2100 HLG curve and Apple camera curve are now implemented by the HLG fixed-function when the LUT-support is turned off.

for now checking-in with LUT_SUPPORT turned off to run the CI tests on the implementation. I'll turn it on later.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
…rameter for the base class. I'm making it explicit, hope that it'll unbreak the builds.

- minor comment fix :)

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
… hope this will un-confuse the compilers.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
…turned off.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
…is stage I'm not sure that turning this warning on the compilers is a good idea or not. Almost all cases are simple unused variabled due to different cmake switches and will be happily optimized out by the compiler without any side-effect. I'm curious to see if that compiler flag helped anyone to catch a bug early on at all.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
…off, turning it back on.

Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
Signed-off-by: cuneyt.ozdas <cuneyt.ozdas@autodesk.com>
@cozdas cozdas marked this pull request as ready for review September 20, 2024 05:22
Signed-off-by: Doug Walker <doug.walker@autodesk.com>
Signed-off-by: Doug Walker <doug.walker@autodesk.com>
@doug-walker doug-walker changed the title Adsk Contrib - Issues #1968 (mirrored-) and #1992 (LUT-free builtins Pt.1) Adsk Contrib - Issues #1968 (mirrored) and #1992 (LUT-free builtins) Sep 21, 2024
Signed-off-by: Doug Walker <doug.walker@autodesk.com>
Copy link
Collaborator

@remia remia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @cozdas and @doug-walker!

Do we need to update the CTF minimum version as well?

@doug-walker
Copy link
Collaborator

@remia , yes absolutely right about the CTF version. I saw you bumped the version in your ACES PR, so my plan is to merge that first to update the CTF version to 2.4 and then we will add the corresponding tests to this PR. There will be some integration work as well in FixedFunction.

Signed-off-by: Doug Walker <doug.walker@autodesk.com>
Signed-off-by: Doug Walker <doug.walker@autodesk.com>
Signed-off-by: Doug Walker <doug.walker@autodesk.com>
Signed-off-by: Doug Walker <doug.walker@autodesk.com>
Signed-off-by: Doug Walker <doug.walker@autodesk.com>
@doug-walker
Copy link
Collaborator

@remia , the merge of the ACES2 PR into this branch was fairly complicated, so I would appreciate it if you could make another pass over this one. And the CTF version check was added.

@remia
Copy link
Collaborator

remia commented Sep 26, 2024

@remia , the merge of the ACES2 PR into this branch was fairly complicated, so I would appreciate it if you could make another pass over this one. And the CTF version check was added.

Looks ok to me!

Copy link
Collaborator

@zachlewis zachlewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Man, this is a lot of work... Cuneyt, you're a maniac. This is awesome.

Looks like you've taken care of #1705 here too, that's cool to see.

I think these "Double Log" and "Gamma Log" FixedFunctions and Builtins are excellent additions, and are quite reasonably named. Having LUT-free alternatives for some of these builtins makes it a lot easier to sling representations of processors around, both in memory and on printed paper.

For the future, I think we need to figure out a better way of documenting FixedFunction parameters.

Fantastic work.

@zachlewis zachlewis merged commit 79e7b7b into AcademySoftwareFoundation:main Sep 26, 2024
25 checks passed
@doug-walker doug-walker deleted the ozdasc/main/BuiltIns_LutFreePt1_Mirrored branch September 26, 2024 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants