According to Jim Carter's website, the 51-54 Okie bushing has the following dimensions: Seal side O.D. = 1.70in, I.D 1.06IN, Drive shaft end O.D. = 1.76IN I.D = 1.34IN TOTAL LENGTH 5.75 IN. Note that the rear end is slightly smaller than the front end, which may be why you hung up before getting it flush with the end of the torque tube.
Here's what they say for the dimensions for the earlier bushings: Seal side O.D. = 1.69in, I.D 1.06IN, Drive shaft end O.D. = 1.73IN I.D = 1.34IN TOTAL LENGTH 5.75 IN. Again, the rear end is smaller than the front end by a few tenths.
Looks like you may have had an early torque tube that never got the pin installed for the bushings. Possibly GM made that torque tube a little smaller and relied on a press fit over the retainer pin for a little while.
You could try some heat on the torque tube to expand it just a bit and get the bushing back out.