you may have also been given the wrong pilot bushing as you shouldn't really have to pound the snot out of it to install it.
recently while installing a pilot bushing in a 283 I ordered a pilot bushing and it was about .023 over size and that is the difference between a b656 and a b656p p/bushing( can't remember the exact measure but that is close). if you don't know the difference you will pound it in and possibly get it a litle off kilter in there
ron