Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compilation Failure due to undefined symbols #276

Closed
dominic-chang opened this issue Jan 26, 2024 · 5 comments
Closed

Compilation Failure due to undefined symbols #276

dominic-chang opened this issue Jan 26, 2024 · 5 comments

Comments

@dominic-chang
Copy link

I'm running julia version 1.9.4 on an M1 mac running Mac OS 13.6. Here is the attached .metallib file: jl_x2z07abwtC.metallib.zip.

I'll try my best to describe what I've noticed. I'm writing a package that has the following method definition.

function (linpol::PowerLawPolarization)(pix::AbstractPixel, geometry::ConeGeometry{T,A}) where {T,A}
    magfield, fluid_velocity, subimgs, profile = geometry.attriributes 

    θs = geometry.opening_angle
    θo = inclination(pix)
    met = metric(pix)
    α, β = screen_coordinate(pix)

    observation = @SVector[zero(T), zero(T), zero(T), zero(T)]

    for n in subimgs
        for isindir in (true, false)
            νθ = cos(θs) < abs(cos(θo)) ? (θo > θs) ⊻ (n % 2 == 1) : !isindir
            rs, νr, _ =  emission_radius(pix, geometry.opening_angle, isindir, n)
            eα, eβ, redshift, lp = polarizationPowerLaw(met, α, β, rs, θs, θo, magfield, fluid_velocity, νr, νθ)
        
            prof = profile(rs)*max(redshift , eps(T))^(3)
            q = T(-(eα^2 - eβ^2)*lp*prof + eps(T))
            u = T(-2*eα*eβ*lp*prof + eps(T))
            i = profile(rs)*T(hypot(q, u))
            nan2zero = x -> isnan(x) ? zero(T) : x
            observation += @SVector[nan2zero(i), nan2zero(q), nan2zero(u), zero(T)]
        end
    end
    return observation
end

Every line in this definition compiles separately, but not all together. So the following two code snippets compile and run without issue:

    magfield, fluid_velocity, subimgs, profile = geometry.attriributes 

    θs = geometry.opening_angle
    θo = inclination(pix)
    met = metric(pix)
    α, β = screen_coordinate(pix)

    observation = @SVector[zero(T), zero(T), zero(T), zero(T)]

    for n in subimgs
        for isindir in (true, false)
            νθ = cos(θs) < abs(cos(θo)) ? (θo > θs) ⊻ (n % 2 == 1) : !isindir
            rs, νr, _ =  hypot(α, β), true, true
            eα, eβ, redshift, lp = polarizationPowerLaw(met, α, β, rs, θs, θo, magfield, fluid_velocity, νr, νθ)
        
            prof = profile(rs)*max(redshift , eps(T))^(3)
            q = T(-(eα^2 - eβ^2)*lp*prof + eps(T))
            u = T(-2*eα*eβ*lp*prof + eps(T))
            i = profile(rs)*T(hypot(q, u))
            nan2zero = x -> isnan(x) ? zero(T) : x
            observation += @SVector[nan2zero(i), nan2zero(q), nan2zero(u), zero(T)]
        end
    end
    return observation
end

and,

    magfield, fluid_velocity, subimgs, profile = geometry.attriributes 

    θs = geometry.opening_angle
    θo = inclination(pix)
    met = metric(pix)
    α, β = screen_coordinate(pix)

    observation = @SVector[zero(T), zero(T), zero(T), zero(T)]

    for n in subimgs
        for isindir in (true, false)
            νθ = cos(θs) < abs(cos(θo)) ? (θo > θs) ⊻ (n % 2 == 1) : !isindir
            rs, νr, _ =  emission_radius(pix, geometry.opening_angle, isindir, n) 
        end
    end
    return observation
end

As far as I can tell, the error is due to an undefined _asn symbol, which I believe is called by the emission_radius method. I however do not have any compilation issues when I broadcast over this method separately.

Hopefully this helps to describe the problem. I can also provide a copy of my package environment if that will help?
@maleadt
Copy link
Member

maleadt commented Jan 29, 2024

Can you post the actual error message? It's unclear what part of the toolchain is failing here.
Also, do you have an MWE?

@dominic-chang
Copy link
Author

dominic-chang commented Jan 30, 2024 •
edited
Loading

I get this error message:

ERROR: Compilation to native code failed; see below for details.
If you think this is a bug, please file an issue and attach /var/folders/69/qsgfh7hj7csg_x4f7kcm2p9r0000gq/T/jl_JwhY1QBopO.metallib.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] link(job::GPUCompiler.CompilerJob, compiled::NamedTuple{(:image, :entry), Tuple{Vector{UInt8}, String}}; return_function::Bool)
    @ Metal ~/.julia/packages/Metal/OchAS/src/compiler/compilation.jl:78
  [3] link(job::GPUCompiler.CompilerJob, compiled::NamedTuple{(:image, :entry), Tuple{Vector{UInt8}, String}})
    @ Metal ~/.julia/packages/Metal/OchAS/src/compiler/compilation.jl:65
  [4] actual_compilation(cache::Dict{Any, Any}, src::Core.MethodInstance, world::UInt64, cfg::GPUCompiler.CompilerConfig{GPUCompiler.MetalCompilerTarget, Metal.MetalCompilerParams}, compiler::typeof(Metal.compile), linker::typeof(Metal.link))
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/execution.jl:132
  [5] cached_compilation(cache::Dict{Any, Any}, src::Core.MethodInstance, cfg::GPUCompiler.CompilerConfig{GPUCompiler.MetalCompilerTarget, Metal.MetalCompilerParams}, compiler::Function, linker::Function)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/execution.jl:103
  [6] macro expansion
    @ ~/.julia/packages/Metal/OchAS/src/compiler/execution.jl:185 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] mtlfunction(f::GPUArrays.var"#broadcast_kernel#38", tt::Type{Tuple{Metal.mtlKernelContext, MtlDeviceMatrix{Float32, 1}, Base.Broadcast.Broadcasted{Metal.MtlArrayStyle{2, Metal.MTL.MTLResourceStorageModePrivate}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, Krang.ElectronSynchrotronPowerLawIntensity, Tuple{Base.Broadcast.Extruded{MtlDeviceMatrix{Krang.IntensityPixel{Float32}, 1}, Tuple{Bool, Bool}, Tuple{Int64, Int64}}, Metal.MtlRefValue{Krang.UnionGeometry{Krang.ConeGeometry{Float32, Tuple{StaticArraysCore.SVector{3, Float32}, StaticArraysCore.SVector{3, Float32}, Tuple{Int64, Int64, Int64}, typeof(profile), Float32, Float32}}, Krang.ConeGeometry{Float32, Tuple{StaticArraysCore.SVector{3, Float32}, StaticArraysCore.SVector{3, Float32}, Tuple{Int64, Int64, Int64}, typeof(profile), Float32, Float32}}}}}}, Int64}}; name::Nothing, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Metal ~/.julia/packages/Metal/OchAS/src/compiler/execution.jl:180
  [9] mtlfunction
    @ ~/.julia/packages/Metal/OchAS/src/compiler/execution.jl:178 [inlined]
 [10] macro expansion
    @ ~/.julia/packages/Metal/OchAS/src/compiler/execution.jl:85 [inlined]
 [11] #launch_heuristic#96
    @ ~/.julia/packages/Metal/OchAS/src/gpuarrays.jl:14 [inlined]
 [12] launch_heuristic
    @ ~/.julia/packages/Metal/OchAS/src/gpuarrays.jl:12 [inlined]
 [13] _copyto!
    @ ~/.julia/packages/GPUArrays/Hd5Sk/src/host/broadcast.jl:56 [inlined]
 [14] copyto!
    @ ~/.julia/packages/GPUArrays/Hd5Sk/src/host/broadcast.jl:37 [inlined]
 [15] copy
    @ ~/.julia/packages/GPUArrays/Hd5Sk/src/host/broadcast.jl:28 [inlined]
 [16] materialize(bc::Base.Broadcast.Broadcasted{Metal.MtlArrayStyle{2, Metal.MTL.MTLResourceStorageModePrivate}, Nothing, Krang.ElectronSynchrotronPowerLawIntensity, Tuple{MtlMatrix{Krang.IntensityPixel{Float32}, Metal.MTL.MTLResourceStorageModePrivate}, Base.RefValue{Krang.UnionGeometry{Krang.ConeGeometry{Float32, Tuple{StaticArraysCore.SVector{3, Float32}, StaticArraysCore.SVector{3, Float32}, Tuple{Int64, Int64, Int64}, typeof(profile), Float32, Float32}}, Krang.ConeGeometry{Float32, Tuple{StaticArraysCore.SVector{3, Float32}, StaticArraysCore.SVector{3, Float32}, Tuple{Int64, Int64, Int64}, typeof(profile), Float32, Float32}}}}}})
    @ Base.Broadcast ./broadcast.jl:873
 [17] macro expansion
    @ ~/.julia/packages/Metal/OchAS/src/utilities.jl:10 [inlined]
 [18] top-level scope
    @ ~/.julia/packages/Metal/OchAS/src/pool.jl:175 [inlined]
 [19] top-level scope
    @ ~/Software/Krang.jl/examples/gpuexample.jl:0

caused by: NSError: Undefined symbols:
  _julia_asn_3237, referenced from: _Z16broadcast_kernel16mtlKernelContext14MtlDeviceArrayI7Float32Li2ELi1EE11BroadcastedI13MtlArrayStyleILi2E39Metal_MTL_MTLResourceStorageModePrivateE5TupleI5OneToI5Int64ES5_IS6_EE36ElectronSynchrotronPowerLawIntensityS4_I8ExtrudedIS0_I14IntensityPixelIS1_ELi2ELi1EES4_I4BoolS10_ES4_IS6_S6_EE11MtlRefValueI13UnionGeometryI12ConeGeometryIS1_S4_I6SArrayIS4_ILi3EES1_Li1ELi3EES14_IS4_ILi3EES1_Li1ELi3EES4_IS6_S6_S6_E7profileS1_S1_EES13_IS1_S4_IS14_IS4_ILi3EES1_Li1ELi3EES14_IS4_ILi3EES1_Li1ELi3EES4_IS6_S6_S6_ES15_S1_S1_EEEEEES6_
  _julia_asn_3237, referenced from: _Z16broadcast_kernel16mtlKernelContext14MtlDeviceArrayI7Float32Li2ELi1EE11BroadcastedI13MtlArrayStyleILi2E39Metal_MTL_MTLResourceStorageModePrivateE5TupleI5OneToI5Int64ES5_IS6_EE36ElectronSynchrotronPowerLawIntensityS4_I8ExtrudedIS0_I14IntensityPixelIS1_ELi2ELi1EES4_I4BoolS10_ES4_IS6_S6_EE11MtlRefValueI13UnionGeometryI12ConeGeometryIS1_S4_I6SArrayIS4_ILi3EES1_Li1ELi3EES14_IS4_ILi3EES1_Li1ELi3EES4_IS6_S6_S6_E7profileS1_S1_EES13_IS1_S4_IS14_IS4_ILi3EES1_Li1ELi3EES14_IS4_ILi3EES1_Li1ELi3EES4_IS6_S6_S6_ES15_S1_S1_EEEEEES6_
 (AGXMetalG13X, code 2)
Stacktrace:
  [1] MTLComputePipelineState(dev::Metal.MTL.MTLDeviceInstance, fun::Metal.MTL.MTLFunctionInstance)
    @ Metal.MTL ~/.julia/packages/Metal/OchAS/lib/mtl/compute_pipeline.jl:60
  [2] link(job::GPUCompiler.CompilerJob, compiled::NamedTuple{(:image, :entry), Tuple{Vector{UInt8}, String}}; return_function::Bool)
    @ Metal ~/.julia/packages/Metal/OchAS/src/compiler/compilation.jl:70
  [3] link(job::GPUCompiler.CompilerJob, compiled::NamedTuple{(:image, :entry), Tuple{Vector{UInt8}, String}})
    @ Metal ~/.julia/packages/Metal/OchAS/src/compiler/compilation.jl:65
  [4] actual_compilation(cache::Dict{Any, Any}, src::Core.MethodInstance, world::UInt64, cfg::GPUCompiler.CompilerConfig{GPUCompiler.MetalCompilerTarget, Metal.MetalCompilerParams}, compiler::typeof(Metal.compile), linker::typeof(Metal.link))
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/execution.jl:132
  [5] cached_compilation(cache::Dict{Any, Any}, src::Core.MethodInstance, cfg::GPUCompiler.CompilerConfig{GPUCompiler.MetalCompilerTarget, Metal.MetalCompilerParams}, compiler::Function, linker::Function)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/execution.jl:103
  [6] macro expansion
    @ ~/.julia/packages/Metal/OchAS/src/compiler/execution.jl:185 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] mtlfunction(f::GPUArrays.var"#broadcast_kernel#38", tt::Type{Tuple{Metal.mtlKernelContext, MtlDeviceMatrix{Float32, 1}, Base.Broadcast.Broadcasted{Metal.MtlArrayStyle{2, Metal.MTL.MTLResourceStorageModePrivate}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, Krang.ElectronSynchrotronPowerLawIntensity, Tuple{Base.Broadcast.Extruded{MtlDeviceMatrix{Krang.IntensityPixel{Float32}, 1}, Tuple{Bool, Bool}, Tuple{Int64, Int64}}, Metal.MtlRefValue{Krang.UnionGeometry{Krang.ConeGeometry{Float32, Tuple{StaticArraysCore.SVector{3, Float32}, StaticArraysCore.SVector{3, Float32}, Tuple{Int64, Int64, Int64}, typeof(profile), Float32, Float32}}, Krang.ConeGeometry{Float32, Tuple{StaticArraysCore.SVector{3, Float32}, StaticArraysCore.SVector{3, Float32}, Tuple{Int64, Int64, Int64}, typeof(profile), Float32, Float32}}}}}}, Int64}}; name::Nothing, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Metal ~/.julia/packages/Metal/OchAS/src/compiler/execution.jl:180
  [9] mtlfunction
    @ ~/.julia/packages/Metal/OchAS/src/compiler/execution.jl:178 [inlined]
 [10] macro expansion
    @ ~/.julia/packages/Metal/OchAS/src/compiler/execution.jl:85 [inlined]
 [11] #launch_heuristic#96
    @ ~/.julia/packages/Metal/OchAS/src/gpuarrays.jl:14 [inlined]
 [12] launch_heuristic
    @ ~/.julia/packages/Metal/OchAS/src/gpuarrays.jl:12 [inlined]
 [13] _copyto!
    @ ~/.julia/packages/GPUArrays/Hd5Sk/src/host/broadcast.jl:56 [inlined]
 [14] copyto!
    @ ~/.julia/packages/GPUArrays/Hd5Sk/src/host/broadcast.jl:37 [inlined]
 [15] copy
    @ ~/.julia/packages/GPUArrays/Hd5Sk/src/host/broadcast.jl:28 [inlined]
 [16] materialize(bc::Base.Broadcast.Broadcasted{Metal.MtlArrayStyle{2, Metal.MTL.MTLResourceStorageModePrivate}, Nothing, Krang.ElectronSynchrotronPowerLawIntensity, Tuple{MtlMatrix{Krang.IntensityPixel{Float32}, Metal.MTL.MTLResourceStorageModePrivate}, Base.RefValue{Krang.UnionGeometry{Krang.ConeGeometry{Float32, Tuple{StaticArraysCore.SVector{3, Float32}, StaticArraysCore.SVector{3, Float32}, Tuple{Int64, Int64, Int64}, typeof(profile), Float32, Float32}}, Krang.ConeGeometry{Float32, Tuple{StaticArraysCore.SVector{3, Float32}, StaticArraysCore.SVector{3, Float32}, Tuple{Int64, Int64, Int64}, typeof(profile), Float32, Float32}}}}}})
    @ Base.Broadcast ./broadcast.jl:873
 [17] macro expansion
    @ ~/.julia/packages/Metal/OchAS/src/utilities.jl:10 [inlined]
 [18] top-level scope
    @ ~/.julia/packages/Metal/OchAS/src/pool.jl:175 [inlined]
 [19] top-level scope
    @ ~/Software/Krang.jl/examples/gpuexample.jl:0

I'll work on seeing if I can produce a MWE

@dominic-chang
Copy link
Author

dominic-chang commented Feb 4, 2024 •
edited
Loading

I think I found the heart of the error. It arrises whenever an array is constructed to be looped over in the function definition. So this causes an error:

arr = MtlArray(zeros(Float32, sze, sze))
function test(pix)
    ans = 0f0
    for n in [0,1,2]
        ans += 1f0
    end
    return sum
end
test.(arr)

while this does not,

arr = MtlArray(zeros(Float32, sze, sze))
function test(pix)
    ans = 0f0
    for n in 0:2
        ans += 1f0
    end
    return sum
end
test.(arr)

@maleadt
Copy link
Member

maleadt commented Feb 16, 2024 •
edited
Loading

I think I found the heart of the error. It arrises whenever an array is constructed to be looped over in the function definition. So this causes an error:

for n in [0,1,2]

You're allocating a CPU array in there, which is unsupported, as the error message tells you:

julia> test.(arr)
ERROR: InvalidIRError: compiling MethodInstance for (::GPUArrays.var"#broadcast_kernel#38")(::Metal.mtlKernelContext, ::MtlDeviceMatrix{typeof(sum), 1}, ::Base.Broadcast.Broadcasted{Metal.MtlArrayStyle{2, Metal.MTL.MTLResourceStorageModePrivate}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(test), Tuple{Base.Broadcast.Extruded{MtlDeviceMatrix{Float32, 1}, Tuple{Bool, Bool}, Tuple{Int64, Int64}}}}, ::Int64) resulted in invalid LLVM IR
Reason: unsupported call through a literal pointer (call to ijl_alloc_array_1d)
Stacktrace:
  [1] Array
    @ ./boot.jl:477
  [2] Array
    @ ./boot.jl:486
  [3] similar
    @ ./abstractarray.jl:884
  [4] similar
    @ ./abstractarray.jl:883
  [5] _array_for
    @ ./array.jl:671
  [6] _array_for
    @ ./array.jl:674
  [7] vect
    @ ./array.jl:126
  [8] test
    @ ./REPL[5]:3
  [9] _broadcast_getindex_evalf
    @ ./broadcast.jl:683
 [10] _broadcast_getindex
    @ ./broadcast.jl:656
 [11] getindex
    @ ./broadcast.jl:610
 [12] broadcast_kernel
    @ ~/.julia/packages/GPUArrays/EoKy0/src/host/broadcast.jl:59

Are you sure you've correctly reduced the error? The Compilation to native code failed issue you originally reported is something we need to fix, but the InvalidIRError your MWE throws is your problem.

Also please report the output of Metal.versioninfo()

@dominic-chang
Copy link
Author

I seem to have made some change and am now unable to reproduce the error. I guess I will be closing this issue since I can't figure out what caused the error in the first place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@maleadt @dominic-chang