apply_deflection: deflected ray direction not propagated on medium boundary exit

[asinghvi17]

Summary

apply_deflection is called correctly during delta tracking and the deflected ray_d is tracked within and between segments. However, when a ray exits a medium through a surface boundary (e.g., a sphere with MediumInterface), the deflected direction is discarded — only escaped rays (no surface hit) use the deflected direction for the environment lookup.

This means apply_deflection cannot produce visible ray-bending effects (like schlieren/gravitational lensing of background geometry or environment maps behind a boundary surface), since almost all rays exit through the boundary surface.

Result of running the MWE

Despite apply_deflection returning Vec3f(0, 0, 1) (straight up) for every ray, stripes are completely undistorted:

Root cause

In src/integrators/volpath/delta-tracking.jl, sample_medium_interaction! (lines ~186–218):

ray_d = result.ray_d   # ← deflected direction from delta tracking

if !work.has_surface_hit
    # Ray escaped — deflected direction IS used ✅
    escaped = VPEscapedRayWorkItem(
        ray_d, work.lambda, work.pixel_index, ...
    )
    push!(escaped_queue, escaped)
else
    # Ray reached surface — original work.ray is used, deflection lost ❌
    push!(hit_surface_queue, VPHitSurfaceWorkItem(work, beta, r_u, r_l))
end

VPHitSurfaceWorkItem(work::VPMediumSampleWorkItem, ...) (in workitems.jl:396–414) reconstructs the hit from work.ray (the original, pre-deflection ray), so:

• The hit point work.hit_pi is computed from the original straight-line ray
• The outgoing direction after material evaluation uses wo = -work.ray.d (original)
• The continuation ray after ThinDielectricMaterial(eta=1.0) transmits in the original direction

Expected behavior

After medium traversal with deflection, the ray exiting through a boundary surface should continue in the deflected direction (and ideally from the deflected exit position).

Minimal reproducer

A custom medium with extreme deflection (returns a fixed direction) still produces no visual change when the medium is bounded by a sphere with MediumInterface:

using GLMakie, RayMakie, Hikari, GeometryBasics, Raycore
using LinearAlgebra: I, norm
using KernelAbstractions

# Minimal deflecting medium — always deflects rays straight up
struct TestDeflectionMedium{T<:AbstractArray{Float32,3}} <: Hikari.Medium
    density::T
    density_res::Vec3{Int}
    max_density::Float32
    bounds::Hikari.Bounds3
    render_to_medium::Mat4f
    medium_to_render::Mat4f
end

function TestDeflectionMedium(density; bounds=Hikari.Bounds3(Point3f(-5f0), Point3f(5f0)))
    nx, ny, nz = size(density)
    TestDeflectionMedium(
        density, Vec3{Int}(nx, ny, nz), Float32(maximum(density)),
        bounds, Mat4f(I), Mat4f(I)
    )
end

Hikari.is_emissive(::TestDeflectionMedium) = false
Hikari.get_template_grid(::TestDeflectionMedium) = Hikari.EmptyMajorantGrid()

function Hikari.sample_point(m::TestDeflectionMedium, media, table, p, λ)
    # Near-zero σ so medium is transparent
    σ = Hikari.SpectralRadiance(0.0001f0)
    Hikari.MediumProperties(σ, σ, Hikari.SpectralRadiance(0f0), 0f0)
end

function Hikari.create_majorant_iterator(m::TestDeflectionMedium, table, ray, t_max, λ)
    t_enter, t_exit = Hikari.ray_bounds_intersect(ray.o, ray.d, m.bounds)
    t_enter = max(t_enter, 0f0)
    t_exit = min(t_exit, t_max)
    t_enter >= t_exit && return Hikari.RayMajorantIterator_homogeneous(0f0, 0f0, Hikari.SpectralRadiance(0f0))
    σ_maj = Hikari.SpectralRadiance(5f0)
    Hikari.RayMajorantIterator_homogeneous(t_enter, t_exit, σ_maj)
end

@Base.propagate_inbounds function Hikari.create_majorant_iterator(
    m::TestDeflectionMedium, table, ray, t_max, λ, tg::M
) where {M<:Hikari.MajorantGrid}
    t_enter, t_exit = Hikari.ray_bounds_intersect(ray.o, ray.d, m.bounds)
    t_enter = max(t_enter, 0f0); t_exit = min(t_exit, t_max)
    res = tg.res
    t_enter >= t_exit && return Hikari.RayMajorantIterator{M}(
        Int32(0), Hikari.SpectralRadiance(0f0), 0f0, 0f0, false, tg,
        (Int32(res[1]), Int32(res[2]), Int32(res[3])),
        (0f0,0f0,0f0), (0f0,0f0,0f0),
        (Int32(0),Int32(0),Int32(0)), (Int32(0),Int32(0),Int32(0)), (Int32(0),Int32(0),Int32(0)))
    Hikari.RayMajorantIterator{M}(
        Int32(1), Hikari.SpectralRadiance(5f0), t_enter, t_exit, false, tg,
        (Int32(res[1]), Int32(res[2]), Int32(res[3])),
        (0f0,0f0,0f0), (0f0,0f0,0f0),
        (Int32(0),Int32(0),Int32(0)), (Int32(0),Int32(0),Int32(0)), (Int32(0),Int32(0),Int32(0)))
end

# EXTREME deflection: always return Vec3f(0,0,1) (straight up)
function Hikari.apply_deflection(::TestDeflectionMedium, p::Point3f, ray_d::Vec3f, dt::Float32)::Vec3f
    Vec3f(0f0, 0f0, 1f0)
end

# --- Build scene ---
density = ones(Float32, 32, 32, 32)
medium = TestDeflectionMedium(density)
mat = Hikari.MediumInterface(Hikari.ThinDielectricMaterial(eta=1.0f0); inside=medium, outside=nothing)

# Stripe environment map
img = Matrix{RGBf}(undef, 512, 512)
for j in 1:512, i in 1:512
    img[i,j] = iseven(div(j-1, 12)) ? RGBf(0.95, 0.95, 0.95) : RGBf(0.05, 0.05, 0.05)
end

fig = Figure(size=(800, 600))
ax = LScene(fig[1,1]; show_axis=false, scenekw=(;
    lights=[Makie.EnvironmentLight(3f0, img)]
))
mesh!(ax, Sphere(Point3f(0,0,0), 5f0*sqrt(3f0));
    color=:black, visible=false, material=mat, transparency=true)

cam = ax.scene.camera_controls
cam.eyeposition[] = Vec3f(0, 25, 1)
cam.lookat[] = Vec3f(0, 0, 0)
cam.upvector[] = Vec3f(0, 0, 1)
cam.fov[] = 35f0
update_cam!(ax.scene, cam)

integrator = Hikari.VolPath(samples=50, max_depth=50)
sensor = Hikari.FilmSensor(iso=100, white_balance=6500)
result = Makie.colorbuffer(ax.scene;
    backend=RayMakie, device=KernelAbstractions.CPU(), integrator=integrator, sensor=sensor)
save("deflection_test.png", result)

Expected: stripes should be completely scrambled (every ray redirected straight up).
Actual: stripes appear perfectly undistorted, identical to a scene with no medium.

Suggested fix

In sample_medium_interaction!, propagate the deflected ray_d into the surface-hit path. This would require:

1. Storing the deflected ray_d and final position in VPHitSurfaceWorkItem
2. Using the deflected direction as the incoming direction for material evaluation (wo = -ray_d instead of wo = -work.ray.d)
3. Optionally updating work.hit_pi to the actual curved-path exit point

This would enable schlieren imaging, atmospheric refraction, and other deflection-based effects that rely on background distortion through a bounded medium.

[asinghvi17]

Result of running the MWE on CPU (KernelAbstractions.CPU()):

The stripes are completely undistorted despite apply_deflection returning Vec3f(0, 0, 1) (straight up) for every ray. The chevron pattern is just the octahedral environment map viewed through a transparent sphere — zero deflection effect visible.

(Image attached below)

[SimonDanisch]

Huuh... Yeah that could be a problem.. I only tested the env map which does show some visible distortion (although not a very strong one).

[asinghvi17]

Yeah I'm trying a couple of things now with media so wanted to make sure it works. Should I PR to your branch?

[SimonDanisch]

Actually, let me merge the PR first so the README also gets an update!