Did you remove the inner race? They sometimes stick to the axle at the backing plate.