Maybe try interpolating between physics frames? See: https://gafferongames.com/post/fix_your_timestep/#the-final-touch