GitHub CI workflow windows build fails #1140

kmturley · 2022-08-15T05:17:44Z

kmturley
Aug 15, 2022

I am working on a cross-platform build of FluidSynth:
https://github.com/kmturley/fluidsynth/blob/master/.github/workflows/all.yml

Build works fine on Linux and Mac, but fails on Windows:
https://github.com/kmturley/fluidsynth/actions/runs/2858518817

The build error it fails on is:

-- Found PkgConfig: C:/ProgramData/chocolatey/bin/pkg-config.exe (found version "0.28") 
-- Checking for modules 'glib-2.0>=2.6.5;gthread-2.0>=2.6.5'
--   No package 'glib-2.0' found
--   No package 'gthread-2.0' found
CMake Error at C:/Program Files/CMake/share/cmake-3.24/Modules/FindPkgConfig.cmake:607 (message):
  A required package was not found
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.24/Modules/FindPkgConfig.cmake:829 (_pkg_check_modules_internal)
  CMakeLists.txt:481 (pkg_check_modules)
-- Configuring incomplete, errors occurred!

See also "D:/a/fluidsynth/fluidsynth/build/CMakeFiles/CMakeOutput.log".

See also "D:/a/fluidsynth/fluidsynth/build/CMakeFiles/CMakeError.log".

I see there is a windows build with commands here:
https://github.com/FluidSynth/fluidsynth/blob/master/.azure/azure-pipelines-win.yml

However it uses environment variables which are not in code, and also doesn't seem to have been run for awhile...Also pkg-config isntallation from choco installs from an old binary from sourceforge, which doesn't seem good.

How can I update my workflow to build FluidSynth for Windows on GitHub CI?
I've tried 35 different combinations of commands, and now feel like I might need some help :)

@derselbst was the last person working on that file.
@3119369616 might also know

Answered by kmturley

Aug 20, 2022

It took a while to navigate issues with cmd/bash, PATH variables, drive letters and missing dependencies. But I finally got it to work sharing the same commands cross-platform and building artifacts which are downloadable:
https://github.com/kmturley/fluidsynth/actions/runs/2893543767

name: FluidSynth All

defaults:
  run:
    shell: bash

on:
  workflow_dispatch:
  pull_request:
  push:

env:
  dir_build: build
  dir_source: .
  type: Release

jobs:
  build:
    name: All platforms
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        include:
          - os: ubuntu-latest
            name: linux
            generator:…

View full answer

derselbst · 2022-08-15T06:39:59Z

derselbst
Aug 15, 2022
Maintainer

We have two Windows CI pipelines and comprehensive docs in the wiki. I'm pretty sure you'll figure it out. I'd recommend to use vcpkg as it's more up to date.

7 replies

derselbst Aug 17, 2022
Maintainer

vcpkg approach is too slow, it takes 30 mins for each build

One can use pipeline caching. Or docker containers... there are many ways around it.

As there are not recent runs I can't see any logs containing urls either

There are plenty of recent runs available, all of them containing the urls. Just click the badges in the readme.

For gtk bundle there are several options on gtk.org, how do I know which is the right one?

Usually, one should use the most recent one.

You seem to be inexperienced in compiling C programs on Windows. What is your motivation to port it to Github CI? Fluidsynth's Azure CI pipelines work fine... except for some randomly occurring HTTP timeouts, which I don't really care about. Why going through all the pain?

kmturley Aug 18, 2022
Author

Sure I can optimize later with caching or containers later, but to just get it to build in the first place it's too slow.
Ah I didn't realize jobs were running on https://dev.azure.com I'd been looking at: https://github.com/FluidSynth/fluidsynth/actions Using the Azure pipeline it is much easier to see the steps and the output. Thanks for the pointer!
Found the urls you use in the Azure pipeline output:

curl -LfsS -o gtk-bundle-dev.zip http://ftp.gnome.org/pub/gnome/binaries/win64/gtk+/2.22/gtk+-bundle_2.22.1-20101229_win64.zip   curl -LfsS -o libsndfile-dev.zip http://www.mega-nerd.com/libsndfile/files/libsndfile-1.0.28-w64.zip   || exit -1 
curl -LfsS -o mingw.zip https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/8.1.0/threads-win32/sjlj/x86_64-8.1.0-release-win32-sjlj-rt_v6-rev0.7z   || exit -1

The libinstpatch binary is not publicly accessible:

Downloading artifact libinstpatch-x64 from: https://dev.azure.com/tommbrt//_apis/resources/Containers/19125121?itemPath=libinstpatch-x64&isShallow=true&api-version=4.1-preview.4

Does that mean I need to manually build a binary of the dependency from the source?
https://github.com/swami/libinstpatch

You are correct, I am inexperience in compiling C programs on Windows. I am however an experienced web developer at creating CI pipelines.

I create and maintain CI pipeline templates for plugin developers, using the main plugin frameworks:
https://github.com/studiorack/studiorack-workflows/tree/main/.github/workflows

My pipeline templates allow you to build any plugin for Mac, Linux and Win in a single pipeline, publishing binaries to GitHub releases in consistent format.

I am trying to build this popular plugin cross-platform:
https://github.com/Birch-san/juicysfplugin

However it fails when trying to compile FluidSynth (as a git submodule).
I need to to be able to build FluidSynth cross-platform to build JuicySF cross platform.

Why am I going through the pain? Because I like the challenge, to learn but also I believe I can share a template which will save others time in the future.

kmturley Aug 18, 2022
Author

I managed to get a successful cross-platform build using different commands on Windows:

    - name: Install Windows dependencies
      if: matrix.os == 'windows-latest'
      shell: cmd
      run: |
        @ECHO ON
        mkdir d:\deps || exit -1
        cd d:\deps || exit -1
        curl -LfsS -o gtk-bundle-dev.zip http://ftp.gnome.org/pub/gnome/binaries/win64/gtk+/2.22/gtk+-bundle_2.22.1-20101229_win64.zip || exit -1
        curl -LfsS -o libsndfile-dev.zip http://www.mega-nerd.com/libsndfile/files/libsndfile-1.0.28-w64.zip || exit -1
        7z x -aos -- gtk-bundle-dev.zip > NUL || exit -1
        7z x -aos -- libsndfile-dev.zip > NUL || exit -1
        REM need to fix the naming of libsndfile otherwise the linker won't find it
        mv lib\libsndfile-1.lib lib\sndfile.lib || exit -1
        mv lib\libsndfile-1.def lib\sndfile.def || exit -1
        DEL /F C:\Strawberry\perl\bin\pkg-config.bat

    - name: Build Windows
      if: matrix.os == 'windows-latest'
      shell: cmd
      run: |
        @ECHO ON
        SET "PATH=d:\deps\bin;%PATH%"
        mkdir build && cd build || exit -1
        cmake -Werror=dev -A x64 -T v142 -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=1 -DNO_GUI=1 -Dwindows-version=0x0A00 -Denable-jack=0 -Denable-pulseaudio=0 -Denable-ladspa=0 -Denable-dbus=0 -Denable-readline=0 -Denable-sdl2=0 -Denable-libinstpatch=0 .. || exit -1
        cmake --build . --config Release --parallel 3 || exit -1
        cmake --build . --config Release --target check || exit -1
        cmake --build . --config Release --target demo || exit -1

The main issues I encountered were:

Using the correct dependency urls
Using the correct shell: cmd instead of bash

Tomorrow I will look at converting the commands to bash and sharing commands between platforms!

derselbst Aug 18, 2022
Maintainer

Does that mean I need to manually build a binary of the dependency from the source? https://github.com/swami/libinstpatch

The windows binaries of libinstpatch are available from its CI pipeline in Azure DevOps. If you don't want fluidsynth to load DLS files, you don't need libinstpatch.

I am trying to build this popular plugin cross-platform: https://github.com/Birch-san/juicysfplugin

But in this case I would recommend to better grab the windows binaries of fluidsynth's stable releases, rather than re-inventing a CI pipeline and potentially building unstable development versions. Also because you're now using very old dependency libraries and because those pipelines need maintenance.

kmturley Aug 20, 2022
Author

juicysfplugin could install fluidsynth as a binary, but wanted to make sure I understood how to build FluidSynth as a git submodule.
I went down the rabbit hole and won't be happy until I solve the problem!

kmturley · 2022-08-20T04:47:56Z

kmturley
Aug 20, 2022
Author

It took a while to navigate issues with cmd/bash, PATH variables, drive letters and missing dependencies. But I finally got it to work sharing the same commands cross-platform and building artifacts which are downloadable:
https://github.com/kmturley/fluidsynth/actions/runs/2893543767

name: FluidSynth All

defaults:
  run:
    shell: bash

on:
  workflow_dispatch:
  pull_request:
  push:

env:
  dir_build: build
  dir_source: .
  type: Release

jobs:
  build:
    name: All platforms
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        include:
          - os: ubuntu-latest
            name: linux
            generator: Unix Makefiles
            flags: -Denable-portaudio=1 -Denable-ladspa=1
            path: ./build/src
          - os: macos-latest
            name: mac
            generator: Unix Makefiles
            flags: -Denable-portaudio=1 -Denable-ladspa=1
            path: ./build/src
          - os: windows-latest
            name: win
            generator: Visual Studio 17 2022
            flags: -Denable-jack=0 -Denable-pulseaudio=0 -Denable-ladspa=0 -Denable-dbus=0 -Denable-readline=0 -Denable-sdl2=0 -Denable-libinstpatch=0
            path: ./build/src/Release

    steps:
    - uses: actions/checkout@v2

    - name: Install Linux dependencies
      if: matrix.os == 'ubuntu-latest'
      run: sudo apt-get update && sudo apt-get install cmake-data cmake libglib2.0-0 libsndfile-dev libasound2-dev libjack-dev portaudio19-dev libsdl2-dev libpulse-dev libdbus-1-dev libsystemd-dev libinstpatch-dev libreadline-dev g++-7 g++-8 clang-7 clang-8 clang-9 clang-10 clang-tidy-10

    - name: Install macOS dependencies
      if: matrix.os == 'macos-latest'
      run: brew install cmake pulseaudio portaudio jack dbus sdl2

    - name: Install Windows dependencies
      if: matrix.os == 'windows-latest'
      run: |
        mkdir D:/deps
        cd D:/deps
        curl -LfsS -o gtk-bundle-dev.zip http://ftp.gnome.org/pub/gnome/binaries/win64/gtk+/2.22/gtk+-bundle_2.22.1-20101229_win64.zip
        curl -LfsS -o libsndfile-dev.zip http://www.mega-nerd.com/libsndfile/files/libsndfile-1.0.28-w64.zip
        7z x -aos -- gtk-bundle-dev.zip
        7z x -aos -- libsndfile-dev.zip
        mv ./lib/libsndfile-1.lib ./lib/sndfile.lib
        mv ./lib/libsndfile-1.def ./lib/sndfile.def
        rm -rf C:/Strawberry/perl/bin/pkg-config*
        echo "/deps/bin" >> $GITHUB_PATH

    - name: Build
      run: |
        cmake \
          -G "${{ matrix.generator }}" \
          -DCMAKE_BUILD_TYPE="${{ env.type }}" \
          -DCMAKE_INSTALL_PREFIX=$HOME/fluidsynth_install \
          -DCMAKE_VERBOSE_MAKEFILE=1 \
          -DNO_GUI=1 \
          ${{ matrix.flags }} \
          -S "${{ env.dir_source }}" \
          -B "${{ env.dir_build }}"
        cmake --build ${{ env.dir_build }} --config ${{ env.type }} --parallel 3
        cmake --build ${{ env.dir_build }} --config ${{ env.type }} --target check
        cmake --build ${{ env.dir_build }} --config ${{ env.type }} --target demo
        cmake --build ${{ env.dir_build }} --config ${{ env.type }} --target install

    - name: List files generated
      run: |
        cd ${{ matrix.path }}
        find "$(pwd)"

    - uses: actions/upload-artifact@v3
      with:
        name: fluidsynth-${{ matrix.name }}
        path: ${{ matrix.path }}

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub CI workflow windows build fails #1140

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 7 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

GitHub CI workflow windows build fails #1140

kmturley Aug 15, 2022

Replies: 2 comments · 7 replies

derselbst Aug 15, 2022 Maintainer

derselbst Aug 17, 2022 Maintainer

kmturley Aug 18, 2022 Author

kmturley Aug 18, 2022 Author

derselbst Aug 18, 2022 Maintainer

kmturley Aug 20, 2022 Author

kmturley Aug 20, 2022 Author

kmturley
Aug 15, 2022

Replies: 2 comments 7 replies

derselbst
Aug 15, 2022
Maintainer

derselbst Aug 17, 2022
Maintainer

kmturley Aug 18, 2022
Author

kmturley Aug 18, 2022
Author

derselbst Aug 18, 2022
Maintainer

kmturley Aug 20, 2022
Author

kmturley
Aug 20, 2022
Author