Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
36 changes: 32 additions & 4 deletions .github/workflows/docker-builds.yml
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ jobs:
--use-conan \
--build-missing

- name: Build base2 system wrapper and dependency Conan packages for ci-baseqt image
- name: Build base2 system wrapper Conan packages for ci-baseqt image
run: |
set -ex
pipenv run aswfdocker \
Expand All @@ -185,6 +185,19 @@ jobs:
build \
--ci-image-type PACKAGE \
--group base2-wrappers \
--version ${{ matrix.version_noclang }} \
--use-conan \
--build-missing

- name: Build base2 system Conan packages for ci-baseqt image
run: |
set -ex
pipenv run aswfdocker \
--repo-uri $GITHUB_REPOSITORY \
--source-branch $GITHUB_REF \
--verbose \
build \
--ci-image-type PACKAGE \
--group base2-1
--version ${{ matrix.version_noclang }} \
--use-conan \
Expand Down Expand Up @@ -231,7 +244,7 @@ jobs:
--version ${{ matrix.version }} \
--push NO

- name: Build vfx1 system wrapper and level 1 Conan packages
- name: Build vfx1 system wrapper Conan packages
run: |
set -ex
pipenv run aswfdocker \
Expand All @@ -241,6 +254,20 @@ jobs:
build \
--ci-image-type PACKAGE \
--group vfx1-wrappers \
--version ${{ matrix.version_noclang }} \
--use-conan \
--build-missing \
--push NO

- name: Build vfx1 level 1 Conan packages
run: |
set -ex
pipenv run aswfdocker \
--repo-uri $GITHUB_REPOSITORY \
--source-branch $GITHUB_REF \
--verbose \
build \
--ci-image-type PACKAGE \
--group vfx1-1 \
--version ${{ matrix.version_noclang }} \
--use-conan \
Expand Down Expand Up @@ -292,7 +319,7 @@ jobs:
--build-missing \
--push NO

- name: Build vfx1 level 5 non-Conan packages
- name: Build vfx1 level 5 Conan packages
run: |
set -ex
pipenv run aswfdocker \
Expand All @@ -301,8 +328,9 @@ jobs:
--verbose \
build \
--ci-image-type PACKAGE \
--group vfx1 \
--group vfx1-5 \
--version ${{ matrix.version_noclang }} \
--use-conan \
--build-missing \
--push NO

Expand Down
81 changes: 81 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,87 @@

All notable changes to this project will be documented in this file.

# 2025-07-XX

These releases no longer build / install libraries in `/usr/local/lib64`, they revert back to the default
`/usr/local/lib` supported by most packages and Conan recipes. This should hopefully be transparent to
consumers of these new images.

- 2024.3 release
- using Conan 2 recipes
- updated versions
- Conan 2.18.1 (was 1.x)
- CUDA 12.6.3 (was 12.6.1)
- ccache 4.9.1 (was 4.8.3)
- CMake 3.31.7 (was 3.27.9)
- Ninja 1.13.1 (was 1.12.1)
- Sonar Cloud 7.1.0.4889 (was 6.2.1.4610)
- Alembic 1.8.8 (was 1.8.6)
- Blosc 1.21.6 (was 1.21.5)
- fmt 11.2.0 (was 11.1.4)
- GLEW 2.2.0 (was 2.1.0)
- GLFW 3.4 (was 3.3.8)
- gtest 1.15.2 (was 1.14.0)
- hdf5 1.14.6
- lcms 2.17 (was 2.16)
- log4cplus 2.1.2 (was 2.1.1)
- minizip-ng 4.0.10 (was 4.0.8)
- pybind11 2.13.6 (was 2.12.0)
- opensubdiv 3.6.1 (was 3.6.0)
- python 3.11.13 (was 3.11.11)
- Qt 6.5.6 (was 6.5.4)
- OIIO 2.5.19.0 (was 2.5.18.0)
- OpenFX 1.5s (new)
- PySide 6.5.6 (was 6.5.4)
- rawtoaces 1.1-rc3 (new)
- 2025.2
- updated versions
- Conan 2.18.1 (was 2.14.0)
- Ninja 1.13.1 (was 1.12.1)
- Sonar Cloud 7.1.0.4889 (was 6.2.1.4610)
- LLVM 19.1.7 (was 19.1.1)
- fmt 11.2.0 (was 11.1.4)
- hdf5 1.14.6
- lcms 2.17 (was 2.16)
- minizip-ng 4.0.10 (was 4.0.8)
- opensubdiv 3.6.1 (was 3.6.0)
- python 3.11.13 (was 3.11.11)
- Qt 6.5.6 (was 6.5.4)
- OIIO 3.0.9.0 (was 3.0.6.1)
- OpenEXR 3.3.5 (was 3.3.3)
- OpenFX 1.5s
- OpenImageIO 3.0.9.1 (was 3.0.6.1)
- OpenVDB 12.0.1 (was 12.0.0)
- OSL 1.14.7.0 (was 1.14.5.1)
- PySide 6.5.6 (was 6.5.4)
- USD 25.05.01 (was 25.05)
- rawtoaces 1.1-rc3 (new)
- 2026.0 draft images
- pre-release for testing purposes, does not yet include final versions of late releasing packages for VFX Platform 2026 (OOCIO, OpenEXR, OpenVDB, OpenSubDiv)
- OpenEXR includes a pre-release of 3.4.x
- see `ci_common6` and `2026` sections of `versions.yaml` for full list of package versions
- to minimize local changes against upstream Conan recipes and avoid spending time fighting build systems which hard code `lib` as the destination directory, all changes related to landing DSOs and support files in `lib64` are reverted
- new Conan package and build images for OpenFX based on conanfile.py from OpenFX repo (Conan Center Index has older version)
- adds dependant Conan package cimg, spdlog
- new [Conan package and build images for rawtoaces](https://github.com/AcademySoftwareFoundation/aswf-docker/issues/273)
- adds dependant Conan packages libraw, jasper, jsonformoderncpp, ceres-solver, eigen , aces_container
- pre-release 1.1-rc3
- Alembic now [built with hdf5 dependency](https://github.com/AcademySoftwareFoundation/aswf-docker/issues/254)
- OpenVDB now [built as a Conan package](https://github.com/AcademySoftwareFoundation/aswf-docker/issues/231)
- OpenImageIO builds with libraw support](https://github.com/AcademySoftwareFoundation/aswf-docker/issues/264) and OpenJPEG/JPEG2000 support
- OpenColorIO Python bindings now being built
- system wrapper Conan packages
- improved wrapper packages no longer declare include directories and libraries they don't include
- query container OS for installed system package version instead of hard coding (pkgconfig / rpm)
- libuhdr Conan package renamed to libultrahdr and updated to match new Conan Center Index package
- blosc Conan package renamed to c-blosc to match Conan Center Index
- Qt now built against harfbuzz text shaping library from vendored Conan package
- aswfdocker utility gets "conandiff" option to show upstream changes to vendored Conan Center Index recipes to help
with keeping recipes up to date
- merge upstream recipe changes, in particular changes for CMake 4 compatibility
- `--keep_source` and `--keep_build` command line options are obsolete, no longer relevant with Conan 2
- install_conanpackages.sh script update to use the Conan full_deploy generator which can deploy all transitive dependencies for a package in one call, mimizing the need to include dependencies in the image.yaml sources for the ci-foo containers

# 2025-05-04

- 2025.1 images
Expand Down
73 changes: 53 additions & 20 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ license.
### Commit Sign-Off

Every commit must be signed off. That is, every commit log message
must include a `Signed-off-by` line (generated, for example, with
`git commit --signoff`), indicating that the committer wrote the
must include a "`Signed-off-by`" line (generated, for example, with
"`git commit --signoff`"), indicating that the committer wrote the
code and has the right to release it under the [Apache-2.0](LICENSE.md)
license. See [Contribution Sign-Off](https://github.com/AcademySoftwareFoundation/tac/blob/main/process/contributing.md#contribution-sign-off)
for more information on this requirement.
Expand All @@ -83,7 +83,7 @@ for more information on this requirement.
### Git Basics

Working with aswf-docker requires understanding a significant amount of
Git and GitHub based terminology. If youre unfamiliar with these
Git and GitHub based terminology. If you're unfamiliar with these
tools or their lingo, please look at the [GitHub
Glossary](https://help.github.com/articles/github-glossary/) or browse
[GitHub Help](https://help.github.com/).
Expand Down Expand Up @@ -113,8 +113,8 @@ contributions should be done on top of it.
After sufficient work is done on the main branch and the aswf-docker
leadership determines that a release is due, we will bump the relevant
internal versioning and tag a commit with the corresponding version
number, e.g. v2.0.1. Each Minor version also has its own Release
Branch, e.g. RB-1.1. This marks a branch of code dedicated to that
number, e.g. v2.0.1. Each Minor version also has its own "Release
Branch", e.g. RB-1.1. This marks a branch of code dedicated to that
Major.Minor version, which allows upstream bug fixes to be
cherry-picked to a given version while still allowing the main
branch to continue forward onto higher versions. This basic repository
Expand All @@ -141,7 +141,7 @@ workflow.

In a typical workflow, you should **fork** the aswf-docker repository to
your account. This creates a copy of the repository under your user
namespace and serves as the home base for your development branches,
namespace and serves as the "home base" for your development branches,
from which you will submit **pull requests** to the upstream
repository to be merged.

Expand Down Expand Up @@ -291,16 +291,19 @@ Use the existing recipes as an example, and borrow from the MIT-licensed
Follow the great instructions there:
[Conan Center Index - How to add Packages](https://github.com/conan-io/conan-center-index/blob/master/docs/how_to_add_packages.md),
but ignore the `config.yml` instructions as the aswfdocker `versions.yaml` already
takes care of listing all the maintained package versions.
takes care of listing all the maintained package versions. If required `config.yml` can still be used when completely different recipes are require for
major package versions, see the `onetbb` package for an example.

Then ensure the ASWF-specific settings are added in the `conanfile.py` such as `python`. This project attempts to minimize local changes to the recipes which originate in the Conan Center Index. Local changes may be marked with a `# ASWF` comment. For simple recipes, the required changes may be as minimal as:
Then ensure the ASWF-specific settings are added in the `conanfile.py` such as `cpython`. This project attempts to minimize local changes to the recipes which originate in the Conan Center Index. Local changes may be marked with a `# ASWF` comment. For simple recipes, the required changes may be as minimal as:

- adding a line to the `layout()` method to adhere to the Enterprise Linux convention of storing 64 bit libraries in the `lib64` directory rather than Conan's default `lib`:
- adding a [SPDX license header](https://spdx.dev/learn/handling-license-info/) where the `From:` URL can be obtained by using the `y` hotkey in the GitHub web UI to obtain a permalink to the version of the Conan recipe file
you are basing hyour local copy on.
```
def layout(self):
cmake_layout(self, src_folder="src")
# ASWF: we want DSOs in lib64
self.cpp.package.libdirs = ["lib64"]
# Copyright (c) Contributors to the conan-center-index Project. All rights reserved.
# Copyright (c) Contributors to the aswf-docker Project. All rights reserved.
# SPDX-License-Identifier: MIT
#
# From: https://github.com/conan-io/conan-center-index/blob/9a66422e07df06d2c502501de6e00b8b1213b563/recipes/opencolorio/all/conanfile.py
```

- adding a `self.name` level to the `copy()` in the `package()` method: when Conan packages are "installed" in the CI build images, they are all flattened together into `/usr/local/` and without this change the license files for all the packages end up in the same directory and can overwrite each other:
Expand All @@ -310,14 +313,13 @@ Then ensure the ASWF-specific settings are added in the `conanfile.py` such as `
copy(self, "LICENSE.md", src=self.source_folder, dst=os.path.join(self.package_folder, "licenses", self.name))
```

- update the calls to `rmdir()` in the `package()` method to point to `lib64`, and
comment out the call that removes the `cmake` directory: we want to be able to use
- comment out the call that removes the `cmake` directory: we want to be able to use
these Conan packages outside the context of Conan, and thus want to retain the
generated `.cmake` files:
```
# ASWF: keep cmake files, delete pkgconfig files in lib64
# rmdir(self, os.path.join(self.package_folder, "lib64", "cmake"))
rmdir(self, os.path.join(self.package_folder, "lib64", "pkgconfig"))
# ASWF: keep cmake files, delete pkgconfig files
# rmdir(self, os.path.join(self.package_folder, "lib", "cmake"))
rmdir(self, os.path.join(self.package_folder, "lib", "pkgconfig"))
```

To help minimize changes to the standard Conan recipes, Conan `profiles` are used to
Expand Down Expand Up @@ -354,6 +356,28 @@ The `--keep-source` and `--keep-build` can help when iterating on the build reci
avoid re-downloading the source, and even keep the previous build artifact.
All regular aswfdocker commands and options work the same with conan or docker packages.

You can use the `aswfdocker conandiff` command to view updates in the upstream Conan Center Index recipes since the last time the local copy was updated. This requires the `From: https://...` header comment to be updated correctly.

```
$ aswfdocker conandiff
Checking conanfile.py files...

Found outdated conanfile.py:
aswf-docker/packages/conan/recipes/minizip-ng/conanfile.py:
Package: minizip-ng
Current SHA: 156d3592a823c0d3d297d8c365eee01f27532a49
Found 1 newer commits:
Commit: 7e056a381694e0fd0b791b9fd06d87d391f461c0
Diff URL: https://github.com/conan-io/conan-center-index/commit/7e056a381694e0fd0b791b9fd06d87d391f461c0
Timestamp: 2024-12-31 14:55:30+00:00
Message:
minizip-ng: add version 4.0.7 (#26112)

* minizip-ng: add version 4.0.7

* rename windows library names in 4.0.7
```

### Docker-only Packages

If a package has no Conan recipe folder its conan package will be skipped at release time.
Expand Down Expand Up @@ -391,7 +415,7 @@ Mutable: true
Reclaimable: true
Shared: false
Size: 93.56GB
Description: cached mount /opt/conan_home/d from exec /bin/sh -c conan create ${ASWF_CONAN_KEEP_SOURCE} ${ASWF_CONAN_KEEP_BUILD} ${ASWF_CONAN_BUILD_MISSING} --profile:all ${ASWF_CONAN_HOME}/.conan2/profiles_${ASWF_PKG_ORG}/${ASWF_CONAN_CHANNEL} --name ${ASWF_PKG_NAME} --version ${ASWF_PKG_VERSION} --user ${ASWF_PKG_ORG} --channel ${ASWF_CONAN_CHANNEL} ${ASWF_CONAN_HOME}/recipes/${ASWF_PKG_NAME} with id "//opt/conan_home/d"
Description: cached mount /opt/conan_home/d from exec /bin/sh -c conan create ${ASWF_CONAN_BUILD_MISSING} --profile:all ${ASWF_CONAN_HOME}/.conan2/profiles_${ASWF_PKG_ORG}/${ASWF_CONAN_CHANNEL} --name ${ASWF_PKG_NAME} --version ${ASWF_PKG_VERSION} --user ${ASWF_PKG_ORG} --channel ${ASWF_CONAN_CHANNEL} ${ASWF_CONAN_HOME}/recipes/${ASWF_PKG_NAME} with id "//opt/conan_home/d"
Usage count: 159
Last used: 14 minutes ago
Type: exec.cachemount
Expand Down Expand Up @@ -421,7 +445,7 @@ where the `b` directory contains the output of the build and the `p` directory w

```
$ ls /var/lib/docker/overlay2/ltu9ddgwws6cblhemncn9uzaj/diff/b/qted742db53b77e/p
bin conaninfo.txt conanmanifest.txt doc include lib64 libexec licenses metatypes mkspecs modules phrasebooks plugins qml resources translations
bin conaninfo.txt conanmanifest.txt doc include lib libexec licenses metatypes mkspecs modules phrasebooks plugins qml resources translations
```

Be careful when trying to modify this cache directory directly, as it is managed by
Expand Down Expand Up @@ -484,6 +508,15 @@ aswfdocker --verbose build -t PACKAGE --group vfx --version 2019 --target usd --
aswfdocker --verbose build -t IMAGE --group vfx --version 2019 --target vfxall --push YES
```

If you are building on system with multiple cores, you may want to set the environment variable:

```
DOCKER_BUILDKIT=1
```

before running `aswfdocker build`, as that will allow Docker BuildKit to run multiple builds in parallel. This is mostly useful when building smaller larger groups of smaller packages: when building individual large packages like Qt or Clang/LLVM,
CMake will run parallel compiles. `DOCKER_BUILDKIT=1` is set when building on GitHub Actions.

### Migrate

`aswfdocker migrate` can migrate images between Docker organizations, should only be used on package images
Expand Down
Loading
Loading