It looks like a bug but I'm not sure is there any setting can avoid this problem.
I set the bullet collider into a small size, but it still hit the other object with the distance it shouldn't be.
Here is the bullet size
And here is the video to show the problem
Customer support service by UserEcho