I'd try and fit a block of wood between the back side of the axle and the flange that the backing plate bolts to. I will make things a bit more firm while your pounding the stud out.