Just to clarify: You can pull the torque tube from the rear housing with out disturbing the pinion ring gear fitment.
If you remove the drive shaft the pinion gear and the shims and the bearings come out with it. It will need to be properly centered for good backlash (you might have to fuss with the shims) not the easiest task but not impossible.
The suggestion about oil levels and seals was to help OP decide if the leak was at the bushing or elsewhere.
The simple fix would be to remove the bushing (leaves the torque tube and drive shaft in place) and address what caused the seal on the bushing to fail.
How ever the bushing can be removed after the drive shaft is removed if that is desired ( there may be many reasons to do this) if you do it this way the old bushings will come out and the rear one may be looked at for excessive wear.
Lots of options and the OP did explain why he thought the seal was bad. I think he did a good job getting to the bushing seal for his fix. Just how he changes that seal still has options. Always more than one way to skin the cat as it were.

Steve H