Nested reducer causes Cilksan crash at runtime and compiler crash when building for Cilkscale #306

ma-chengyuan · 2025-01-13T08:51:19Z

Describe the bug

Code using multiple reducers in nested cilk_for loops compilers fine without instrumentation. However when compiled with Cilksan the executable crashes at runtime and the compiler crashes when compiled with Cilkscale.

Expected behavior

Neither the compiler nor the compiled executable should crash.

OpenCilk version

Built from source:

opencilk-project: ef3cdc8
cheetah: b71934ce4dbb5db3e4369d2cbc54d1e7fb9a1c3f
productivity-tools: 393e6849624ad4c9f30954309405afbf931f60e5
infrastructure: ee22572e020e96b0ae1d05c046c201281ee713e3

System information

OS: Arch Linux (as of Jan 13 2025)
CPU: AMD Ryzen 9 5900HX with Radeon Graphics

Steps to reproduce (include relevant output)

Copy the working example code below to nested_reducer_mwe.cpp
Build with following:

#!/bin/bash

cilk_path=/opt/opencilk-2/bin/clang++

$cilk_path -std=c++20 -O3 -g -fopencilk -fsanitize=cilk nested_reducer_mwe.cpp -o nested_reducer_mwe_cilksan
./nested_reducer_mwe_cilksan

$cilk_path -std=c++20 -O3 -g -fopencilk -fcilktool=cilkscale nested_reducer_mwe.cpp -o nested_reducer_mwe_cilkscale
./nested_reducer_mwe_cilkscale

nested_reducer_mwe_cilksan fails with

Running Cilksan race detector.
free(): invalid pointer

nested_reducer_mwe_cilkscale compilation fails with

clang-18: /home/chengyuan/Projects/opencilk/opencilk-project/llvm/include/llvm/IR/DataLayout.h:673: llvm::TypeSize llvm::DataLayout::getTypeSizeInBits(llvm::Type*) const: Assertion `Ty->isSized() && "Cannot getTypeInfo() on a type that is unsized!"' failed.
PLEASE submit a bug report to https://github.com/OpenCilk/opencilk-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: /home/chengyuan/Projects/opencilk/build/bin/clang-18 -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -dumpdir nested_reducer_mwe_cilkscale- -disable-free -clear-ast-before-backend -main-file-name nested_reducer_mwe.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=none -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debug-info-kind=constructor -dwarf-version=5 -debugger-tuning=gdb -fdebug-compilation-dir=/home/chengyuan/Projects/meng/gpu/cilk_bugs -fcoverage-compilation-dir=/home/chengyuan/Projects/meng/gpu/cilk_bugs -resource-dir /home/chengyuan/Projects/opencilk/build/lib/clang/18 -internal-isystem /usr/lib64/gcc/x86_64-pc-linux-gnu/14.2.1/../../../../include/c++/14.2.1 -internal-isystem /usr/lib64/gcc/x86_64-pc-linux-gnu/14.2.1/../../../../include/c++/14.2.1/x86_64-pc-linux-gnu -internal-isystem /usr/lib64/gcc/x86_64-pc-linux-gnu/14.2.1/../../../../include/c++/14.2.1/backward -internal-isystem /home/chengyuan/Projects/opencilk/build/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/lib64/gcc/x86_64-pc-linux-gnu/14.2.1/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -std=c++20 -fdeprecated-macro -ferror-limit 19 -fopencilk --opencilk-abi-bitcode=/home/chengyuan/Projects/opencilk/build/lib/clang/18/lib/x86_64-unknown-linux-gnu/libopencilk-abi.bc -fcilktool=cilkscale -fgnuc-version=4.2.1 -fno-implicit-modules -fskip-odr-check-in-gmf -fcxx-exceptions -fexceptions -fcolor-diagnostics -vectorize-loops -vectorize-slp -stripmine-loops -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/nested_reducer_mwe-dee360.o -x c++ nested_reducer_mwe.cpp
1.      <eof> parser at end of file
2.      Optimizer
 #0 0x000055d15fc8861c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x285e61c)
 #1 0x000055d15fc855bb SignalHandler(int) Signals.cpp:0:0
 #2 0x000077a0b235d1d0 (/usr/lib/libc.so.6+0x3d1d0)
 #3 0x000077a0b23b63f4 (/usr/lib/libc.so.6+0x963f4)
 #4 0x000077a0b235d120 raise (/usr/lib/libc.so.6+0x3d120)
 #5 0x000077a0b23444c3 abort (/usr/lib/libc.so.6+0x244c3)
 #6 0x000077a0b23443df (/usr/lib/libc.so.6+0x243df)
 #7 0x000077a0b2355177 (/usr/lib/libc.so.6+0x35177)
 #8 0x000055d15e33963a (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0xf0f63a)
 #9 0x000055d15fcbe90a void std::__introsort_loop<llvm::Value**, long, __gnu_cxx::__ops::_Iter_comp_iter<llvm::fixupInputSet(llvm::Function&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u> const&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u>&)::'lambda'(llvm::Value const*, llvm::Value const*)>>(llvm::Value**, llvm::Value**, long, __gnu_cxx::__ops::_Iter_comp_iter<llvm::fixupInputSet(llvm::Function&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u> const&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u>&)::'lambda'(llvm::Value const*, llvm::Value const*)>) LoweringUtils.cpp:0:0
#10 0x000055d15fcc691f llvm::fixupInputSet(llvm::Function&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u> const&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u>&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x289c91f)
#11 0x000055d15fcc6f2f llvm::outlineTaskFrame(llvm::Spindle*, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u>&, llvm::SmallVectorImpl<llvm::Value*>&, llvm::Module*, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex<false>>>&, llvm::TapirTarget::ArgStructMode, llvm::Type*, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex<false>>>&, llvm::OutlineAnalysis&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x289cf2f)
#12 0x000055d15fcfa13d TapirToTargetImpl::outlineAllTasks(llvm::Function&, llvm::SmallVectorImpl<llvm::Spindle*>&, llvm::OutlineAnalysis&, llvm::TaskInfo&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x28d013d)
#13 0x000055d15fcfcbea TapirToTargetImpl::processFunction(llvm::Function&, llvm::SmallVectorImpl<llvm::Function*>&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x28d2bea)
#14 0x000055d15fcff41a TapirToTargetImpl::run() (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x28d541a)
#15 0x000055d15fcff988 llvm::TapirToTargetPass::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x28d5988)
#16 0x000055d1611c619f llvm::detail::PassModel<llvm::Module, llvm::TapirToTargetPass, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x3d9c19f)
#17 0x000055d15f714011 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x22ea011)
#18 0x000055d15ffe187a (anonymous namespace)::EmitAssemblyHelper::RunOptimizationPipeline(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>&, std::unique_ptr<llvm::ToolOutputFile, std::default_delete<llvm::ToolOutputFile>>&, clang::BackendConsumer*) BackendUtil.cpp:0:0
#19 0x000055d15ffe54c1 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x2bbb4c1)
#20 0x000055d160643e04 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x3219e04)
#21 0x000055d1623d07a9 clang::ParseAST(clang::Sema&, bool, bool) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x4fa67a9)
#22 0x000055d160909369 clang::FrontendAction::Execute() (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x34df369)
#23 0x000055d1608823f6 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x34583f6)
#24 0x000055d1609e0878 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0x35b6878)
#25 0x000055d15e2e8a86 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0xebea86)
#26 0x000055d15e2e0617 ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) driver.cpp:0:0
#27 0x000055d15e2e4335 clang_main(int, char**, llvm::ToolContext const&) (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0xeba335)
#28 0x000055d15e1fc194 main (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0xdd2194)
#29 0x000077a0b2345e08 (/usr/lib/libc.so.6+0x25e08)
#30 0x000077a0b2345ecc __libc_start_main (/usr/lib/libc.so.6+0x25ecc)
#31 0x000055d15e2df1f5 _start (/home/chengyuan/Projects/opencilk/build/bin/clang-18+0xeb51f5)
clang++: error: unable to execute command: Aborted (core dumped)
clang++: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 18.1.8 (https://github.com/OpenCilk/opencilk-project ef3cdc8bd3ece700a9109ed4c006e90e8a788a84)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/chengyuan/Projects/opencilk/build/bin

Working example code

The following code was isolated from a larger codebase:

#include <array>
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <random>

struct Scalar {
    constexpr static uint64_t PRIME = 0xffffffff00000001ull;

    // Scalar() { /* Deliberately skip initialization of _raw to allow more
    //               aggresive optimization */
    // }

    Scalar() = default;

    explicit Scalar(uint64_t raw) : _raw{raw} {}

    explicit operator uint64_t() const { return _raw; }

    auto operator+(Scalar other) const -> Scalar {
        const uint64_t sum = _raw + other._raw;
        const Scalar ret{
            sum < _raw || sum < other._raw || sum >= PRIME ? sum - PRIME : sum};
        return ret;
    }

    auto operator-(Scalar other) const -> Scalar {
        const uint64_t diff = _raw - other._raw;
        const Scalar ret{(diff > _raw) ? diff + PRIME : diff};
        return ret;
    }

    auto operator*(Scalar other) const -> Scalar {
        // Start by carrying out an ordinary 64x64->128 bit multiplication
        const uint32_t a0 = _raw, a1 = _raw >> 32;
        const uint32_t b0 = other._raw, b1 = other._raw >> 32;
        const uint64_t p0 = static_cast<uint64_t>(a0) * b0,
                       p1 = static_cast<uint64_t>(a0) * b1,
                       p2 = static_cast<uint64_t>(a1) * b0,
                       p3 = static_cast<uint64_t>(a1) * b1;
        const uint32_t cy = ((p0 >> 32) + static_cast<uint32_t>(p1) +
                             static_cast<uint32_t>(p2)) >>
                            32;
        const uint64_t x = p0 + (p1 << 32) + (p2 << 32),
                       y = p3 + (p1 >> 32) + (p2 >> 32) + cy;
        // Store result in 4 32-bit words
        const uint32_t c0 = x, c1 = x >> 32, c2 = y, c3 = y >> 32;
        // Now perform reduction: modulus is phi^2 - phi + 1 where phi = 2^32
        //   ab = c0 + c1*phi + c2*phi^2 + c3*phi^3
        // Exploit phi^2 = phi-1 and phi^3 = phi * (phi-1) = (phi-1) - phi = -1
        //   ab = c0 + c1*phi + c2*(phi-1) - c3
        //      = (c0-c2-c3) + (c1+c2)*phi
        const Scalar ret = (Scalar(c0) - Scalar(c2) - Scalar(c3)) +
                           (Scalar(static_cast<uint64_t>(c1) << 32) +
                            Scalar(static_cast<uint64_t>(c2) << 32));
        return ret;
    }

    auto operator==(Scalar other) const -> bool { return _raw == other._raw; }

    auto operator!=(Scalar other) const -> bool { return _raw != other._raw; }

    // No comparison operators as they make no sense for finite fields

    auto operator+=(Scalar other) -> Scalar & { return *this = *this + other; }

    auto operator-=(Scalar other) -> Scalar & { return *this = *this - other; }

    auto operator*=(Scalar other) -> Scalar & { return *this = *this * other; }

    template <typename RNG> inline static auto random(RNG &rng) -> Scalar {
        static std::uniform_int_distribution<uint64_t> dist(0, PRIME - 1);
        return Scalar{dist(rng)};
    }

    auto is_valid() const -> bool { return _raw < PRIME; }

  private:
    uint64_t _raw;
};

static inline void zero_scalar(void *view) {
    *reinterpret_cast<Scalar *>(view) = Scalar{0};
}

static inline void add_scalar(void *left, void *right) {
    *reinterpret_cast<Scalar *>(left) += *reinterpret_cast<Scalar *>(right);
}

using ScalarAddReducer = Scalar cilk_reducer(zero_scalar, add_scalar);

auto reduce_with_cilk(Scalar **as, Scalar **bs, Scalar *c, Scalar *coeffs,
                      size_t n, size_t m) -> std::array<Scalar, 3> {
    ScalarAddReducer p0{0}, p2{0}, p3{0};
    cilk_for (size_t i = 0; i < n; i++) {
        // Obtain dense representations of the polynomials
        const Scalar *a = as[i];
        const Scalar *b = bs[i];
        const size_t half = m / 2;

        ScalarAddReducer lp0{0}, lp2{0}, lp3{0};
        cilk_for (size_t j = 0; j < half; j++) {
            lp0 += a[j] * b[j] * c[j];
            const Scalar a2 = a[j + half] + a[j + half] - a[j],
                         b2 = b[j + half] + b[j + half] - b[j],
                         c2 = c[j + half] + c[j + half] - c[j];
            lp2 += a2 * b2 * c2;
            const Scalar a3 = a2 + a[j + half] - a[j],
                         b3 = b2 + b[j + half] - b[j],
                         c3 = c2 + c[j + half] - c[j];
            lp3 += a3 * b3 * c3;
        }

        p0 += coeffs[i] * lp0;
        p2 += coeffs[i] * lp2;
        p3 += coeffs[i] * lp3;
    }
    return {p0, p2, p3};
}

auto reduce_serial(Scalar **as, Scalar **bs, Scalar *c, Scalar *coeffs,
                   size_t n, size_t m) -> std::array<Scalar, 3> {
    Scalar p0{0}, p2{0}, p3{0};
    for (size_t i = 0; i < n; i++) {
        // Obtain dense representations of the polynomials
        const Scalar *a = as[i];
        const Scalar *b = bs[i];
        const size_t half = m / 2;

        Scalar lp0{0}, lp2{0}, lp3{0};
        for (size_t j = 0; j < half; j++) {
            lp0 += a[j] * b[j] * c[j];
            const Scalar a2 = a[j + half] + a[j + half] - a[j],
                         b2 = b[j + half] + b[j + half] - b[j],
                         c2 = c[j + half] + c[j + half] - c[j];
            lp2 += a2 * b2 * c2;
            const Scalar a3 = a2 + a[j + half] - a[j],
                         b3 = b2 + b[j + half] - b[j],
                         c3 = c2 + c[j + half] - c[j];
            lp3 += a3 * b3 * c3;
        }

        p0 += coeffs[i] * lp0;
        p2 += coeffs[i] * lp2;
        p3 += coeffs[i] * lp3;
    }
    return {p0, p2, p3};
}

auto main() -> int {
    const size_t N = 12;
    const size_t M = 128;

    std::mt19937_64 rng{42};

    Scalar *as[N];
    Scalar *bs[N];
    Scalar c[M];
    Scalar coeffs[N];

    for (size_t i = 0; i < N; i++) {
        as[i] = new Scalar[M];
        bs[i] = new Scalar[M];
        coeffs[i] = Scalar::random(rng);
        for (size_t j = 0; j < M; j++) {
            as[i][j] = Scalar::random(rng);
            bs[i][j] = Scalar::random(rng);
        }
    }
    for (size_t i = 0; i < M; i++) {
        c[i] = Scalar::random(rng);
    }

    const auto res_cilk = reduce_with_cilk(as, bs, c, coeffs, N, M);
    const auto res_serial = reduce_serial(as, bs, c, coeffs, N, M);
    if (res_cilk != res_serial) {
        printf("res_cilk = %lu, %lu, %lu\n", static_cast<uint64_t>(res_cilk[0]),
               static_cast<uint64_t>(res_cilk[1]),
               static_cast<uint64_t>(res_cilk[2]));
        printf("res_serial = %lu, %lu, %lu\n",
               static_cast<uint64_t>(res_serial[0]),
               static_cast<uint64_t>(res_serial[1]),
               static_cast<uint64_t>(res_serial[2]));
    }
    for (size_t i = 0; i < N; i++) {
        delete[] as[i];
        delete[] bs[i];
    }
    return 0;
}

// NOLINTEND

Additional comments

In the 2.1 release version, the compiled code (without Cilksan / Cilkscale) also produced non-deterministic output, but it seems to have been fixed since then.

The text was updated successfully, but these errors were encountered:

VoxSciurorum · 2025-01-13T11:17:46Z

I have reproduced both bugs on other OSes as well, MacOS/x86-64 and FreeBSD/arm64.

VoxSciurorum · 2025-01-13T17:50:43Z

This bug was apparently introduced by 59ca238 ("Properly enclose the body of a Tapir loop within a taskframe...").

neboat · 2025-01-15T00:33:57Z

I think I have fixes for these bugs, as well as another bug I found in the course of debugging. We'll work on a PR with a fix and incorporating the fix into the next release.

ma-chengyuan · 2025-01-17T09:55:45Z

Thanks for the quick fixes! Are the PRs still a work in progress? After building from the PR branch (both the compiler and cilktools), I am still getting the following error:

Running Cilksan race detector.
./build.sh: line 7: 69877 Segmentation fault      (core dumped) ./nested_reducer_mwe_cilksan
./build.sh: line 10: 69903 Segmentation fault      (core dumped) ./nested_reducer_mwe_cilkscale

neboat · 2025-01-17T12:00:34Z

We are in the process of rebasing the OpenCilk compiler onto LLVM 19. I believe the PR for the compiler fix is based on that rebased compiler. Are you using that LLVM-19-based compiler? In other words, what version do you see when you run clang --version ?

If you're using the LLVM-19-based version of OpenCilk, make sure you update your copy of cheetah as well. I just merged a fix to to cheetah that is necessary for the new version of the compiler. Without that update, you're likely to see segfaults.

Let me know if updating cheetah resolves the issue on your end.

ma-chengyuan · 2025-01-18T13:22:54Z

Thank you for the suggestion. Updating cheetah and rebuilding indeed fixes the problem above. The MWE above now compiles and runs without issue.

However, I am still getting a compiler crash when building my original codebase with Cilkscale:

/home/chengyuan/Projects/opencilk/build/bin/clang++ -DBOOST_STACKTRACE_NO_LIB -DBOOST_STACKTRACE_STATIC_LINK -DBOOST_STACKTRACE_USE_BACKTRACE -DSPDLOG_COMPILED_LIB -I/home/chengyuan/Projects/meng/gpu/include -I/home/chengyuan/Projects/meng/gpu/libmerlin -I/home/chengyuan/Projects/meng/gpu/build_cilkscale/_deps/xxhash-src/cmake_unofficial/.. -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/stacktrace/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/config/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/container_hash/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/describe/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/mp11/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/core/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/assert/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/static_assert/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/throw_exception/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/predef/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/winapi/include -isystem /home/chengyuan/.cache/CPM/fmt/cccb77ae9609d2768ed80dd42cec54f77b1f1455/include -isystem /home/chengyuan/.cache/CPM/spdlog/12508e60ef742a41db4d8db5a4334e10142c995d/include -isystem /home/chengyuan/.cache/CPM/catch2/1b2c19d23bca710b0cab412b221d18c1a5b6e30e/src/catch2/.. -isystem /home/chengyuan/Projects/meng/gpu/build_cilkscale/_deps/catch2-build/generated-includes -isystem /home/chengyuan/.cache/CPM/json/798e0374658476027d9723eeb67a262d0f3c8308/include -isystem /home/chengyuan/.cache/CPM/args/d52a8c14bc936001deccf6f63d0def34b0daaf3a -O2 -g -DNDEBUG -std=gnu++20 -fopencilk -DCILK_ENABLED -Rpass-analysis=loop-spawning-ti -fcilktool=cilkscale -MD -MT CMakeFiles/cu-zksnark.dir/src/spark.cpp.o -MF CMakeFiles/cu-zksnark.dir/src/spark.cpp.o.d -o CMakeFiles/cu-zksnark.dir/src/spark.cpp.o -c /home/chengyuan/Projects/meng/gpu/src/spark.cpp
clang++: /home/chengyuan/Projects/opencilk/opencilk-project/llvm/include/llvm/IR/DataLayout.h:673: llvm::TypeSize llvm::DataLayout::getTypeSizeInBits(llvm::Type*) const: Assertion `Ty->isSized() && "Cannot getTypeInfo() on a type that is unsized!"' failed.
PLEASE submit a bug report to https://github.com/OpenCilk/opencilk-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: /home/chengyuan/Projects/opencilk/build/bin/clang++ -DBOOST_STACKTRACE_NO_LIB -DBOOST_STACKTRACE_STATIC_LINK -DBOOST_STACKTRACE_USE_BACKTRACE -DSPDLOG_COMPILED_LIB -I/home/chengyuan/Projects/meng/gpu/include -I/home/chengyuan/Projects/meng/gpu/libmerlin -I/home/chengyuan/Projects/meng/gpu/build_cilkscale/_deps/xxhash-src/cmake_unofficial/.. -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/stacktrace/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/config/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/container_hash/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/describe/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/mp11/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/core/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/assert/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/static_assert/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/throw_exception/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/predef/include -I/home/chengyuan/.cache/CPM/boost/92bb54148e065c73032bd7e34d267d15b27c1cc2/libs/winapi/include -isystem /home/chengyuan/.cache/CPM/fmt/cccb77ae9609d2768ed80dd42cec54f77b1f1455/include -isystem /home/chengyuan/.cache/CPM/spdlog/12508e60ef742a41db4d8db5a4334e10142c995d/include -isystem /home/chengyuan/.cache/CPM/catch2/1b2c19d23bca710b0cab412b221d18c1a5b6e30e/src/catch2/.. -isystem /home/chengyuan/Projects/meng/gpu/build_cilkscale/_deps/catch2-build/generated-includes -isystem /home/chengyuan/.cache/CPM/json/798e0374658476027d9723eeb67a262d0f3c8308/include -isystem /home/chengyuan/.cache/CPM/args/d52a8c14bc936001deccf6f63d0def34b0daaf3a -O2 -g -DNDEBUG -std=gnu++20 -fopencilk -DCILK_ENABLED -Rpass-analysis=loop-spawning-ti -fcilktool=cilkscale -MD -MT CMakeFiles/cu-zksnark.dir/src/spark.cpp.o -MF CMakeFiles/cu-zksnark.dir/src/spark.cpp.o.d -o CMakeFiles/cu-zksnark.dir/src/spark.cpp.o -c /home/chengyuan/Projects/meng/gpu/src/spark.cpp
1.      <eof> parser at end of file
2.      Optimizer
3.      Running pass "tapir2target" on module "/home/chengyuan/Projects/meng/gpu/src/spark.cpp"
 #0 0x00005f5015f398dc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x29808dc)
 #1 0x00005f5015f36d74 llvm::sys::CleanupOnSignal(unsigned long) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x297dd74)
 #2 0x00005f5015e7b200 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
 #3 0x00007d912301e1d0 (/usr/lib/libc.so.6+0x3d1d0)
 #4 0x00007d91230773f4 (/usr/lib/libc.so.6+0x963f4)
 #5 0x00007d912301e120 raise (/usr/lib/libc.so.6+0x3d120)
 #6 0x00007d91230054c3 abort (/usr/lib/libc.so.6+0x244c3)
 #7 0x00007d91230053df (/usr/lib/libc.so.6+0x243df)
 #8 0x00007d9123016177 (/usr/lib/libc.so.6+0x35177)
 #9 0x00005f501457b44a (/home/chengyuan/Projects/opencilk/build/bin/clang+++0xfc244a)
#10 0x00005f5015f6f4c3 void std::__insertion_sort<llvm::Value**, __gnu_cxx::__ops::_Iter_comp_iter<llvm::fixupInputSet(llvm::Function&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u> const&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u>&)::'lambda'(llvm::Value const*, llvm::Value const*)>>(llvm::Value**, llvm::Value**, __gnu_cxx::__ops::_Iter_comp_iter<llvm::fixupInputSet(llvm::Function&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u> const&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u>&)::'lambda'(llvm::Value const*, llvm::Value const*)>) LoweringUtils.cpp:0:0
#11 0x00005f5015f784c0 llvm::fixupInputSet(llvm::Function&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u> const&, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u>&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x29bf4c0)
#12 0x00005f5015f788e4 llvm::outlineTaskFrame(llvm::Spindle*, llvm::SetVector<llvm::Value*, llvm::SmallVector<llvm::Value*, 0u>, llvm::DenseSet<llvm::Value*, llvm::DenseMapInfo<llvm::Value*, void>>, 0u>&, llvm::SmallVectorImpl<llvm::Value*>&, llvm::Module*, llvm::CloneFunctionChangeType, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex<false>>>&, llvm::TapirTarget::ArgStructMode, llvm::Type*, llvm::ValueMap<llvm::Value const*, llvm::WeakTrackingVH, llvm::ValueMapConfig<llvm::Value const*, llvm::sys::SmartMutex<false>>>&, llvm::OutlineAnalysis&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x29bf8e4)
#13 0x00005f5015faba9c TapirToTargetImpl::outlineAllTasks(llvm::Function&, llvm::SmallVectorImpl<llvm::Spindle*>&, llvm::OutlineAnalysis&, llvm::TaskInfo&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x29f2a9c)
#14 0x00005f5015fae69e TapirToTargetImpl::processFunction(llvm::Function&, llvm::SmallVectorImpl<llvm::Function*>&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x29f569e)
#15 0x00005f5015fb0eaa TapirToTargetImpl::run() (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x29f7eaa)
#16 0x00005f5015fb1418 llvm::TapirToTargetPass::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x29f8418)
#17 0x00005f50175885ff llvm::detail::PassModel<llvm::Module, llvm::TapirToTargetPass, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x3fcf5ff)
#18 0x00005f50159b471d llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x23fb71d)
#19 0x00005f50162beb58 (anonymous namespace)::EmitAssemblyHelper::RunOptimizationPipeline(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>&, std::unique_ptr<llvm::ToolOutputFile, std::default_delete<llvm::ToolOutputFile>>&, clang::BackendConsumer*) BackendUtil.cpp:0:0
#20 0x00005f50162c2840 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x2d09840)
#21 0x00005f501698c01d clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x33d301d)
#22 0x00005f50188a15ec clang::ParseAST(clang::Sema&, bool, bool) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x52e85ec)
#23 0x00005f5016c73ad9 clang::FrontendAction::Execute() (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x36baad9)
#24 0x00005f5016bf29de clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x36399de)
#25 0x00005f5016d56d20 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x379dd20)
#26 0x00005f5014507924 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0xf4e924)
#27 0x00005f50144ffdc0 ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) driver.cpp:0:0
#28 0x00005f50169d74a9 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const::'lambda'()>(long) Job.cpp:0:0
#29 0x00005f5015e7b6a2 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x28c26a2)
#30 0x00005f50169d9916 clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (.part.0) Job.cpp:0:0
#31 0x00005f501699782c clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x33de82c)
#32 0x00005f50169982be clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x33df2be)
#33 0x00005f50169a4874 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0x33eb874)
#34 0x00005f5014503f06 clang_main(int, char**, llvm::ToolContext const&) (/home/chengyuan/Projects/opencilk/build/bin/clang+++0xf4af06)
#35 0x00005f50143fb9e4 main (/home/chengyuan/Projects/opencilk/build/bin/clang+++0xe429e4)
#36 0x00007d9123006e08 (/usr/lib/libc.so.6+0x25e08)
#37 0x00007d9123006ecc __libc_start_main (/usr/lib/libc.so.6+0x25ecc)
#38 0x00005f50144ff3b5 _start (/home/chengyuan/Projects/opencilk/build/bin/clang+++0xf463b5)
clang++: error: clang frontend command failed with exit code 134 (use -v to see invocation)
clang version 19.1.7 (https://github.com/neboat/opencilk-project.git e929b19f1ca3426871e22a5843cc9e5725894576)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/chengyuan/Projects/opencilk/build/bin
Build config: +assertions
clang++: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang++: note: diagnostic msg: /tmp/spark-6e197b.cpp
clang++: note: diagnostic msg: /tmp/spark-6e197b.sh
clang++: note: diagnostic msg: 

********************

spark-6e197b.zip

Should I open a separate issue for this?

neboat · 2025-01-18T14:27:20Z

Thanks for the new report. We can just use this issue thread while we investigate.

ma-chengyuan added the bug Something isn't working label Jan 13, 2025

This was referenced Jan 15, 2025

Fix compiler crashes when instrumenting programs with nested Cilk contexts and nested reducers #309

Open

[cilksan] Fix logic for combining reducer views to handle cases where… OpenCilk/productivity-tools#53

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nested reducer causes Cilksan crash at runtime and compiler crash when building for Cilkscale #306

Nested reducer causes Cilksan crash at runtime and compiler crash when building for Cilkscale #306

ma-chengyuan commented Jan 13, 2025

VoxSciurorum commented Jan 13, 2025

VoxSciurorum commented Jan 13, 2025

neboat commented Jan 15, 2025

ma-chengyuan commented Jan 17, 2025

neboat commented Jan 17, 2025

ma-chengyuan commented Jan 18, 2025

neboat commented Jan 18, 2025

Nested reducer causes Cilksan crash at runtime and compiler crash when building for Cilkscale #306

Nested reducer causes Cilksan crash at runtime and compiler crash when building for Cilkscale #306

Comments

ma-chengyuan commented Jan 13, 2025

Describe the bug

Expected behavior

OpenCilk version

System information

Steps to reproduce (include relevant output)

Working example code

Additional comments

VoxSciurorum commented Jan 13, 2025

VoxSciurorum commented Jan 13, 2025

neboat commented Jan 15, 2025

ma-chengyuan commented Jan 17, 2025

neboat commented Jan 17, 2025

ma-chengyuan commented Jan 18, 2025

neboat commented Jan 18, 2025