After much shenanigans, some broken tools, some nasty bread, and several messes, the pilot bushing is out.
I test fit the new one and it fit my input shaft nicely. It tapped right in and the clutch alignment tool fit easily in it.
Being mildly paranoid from Kevin's comment above, I checked with the input shaft and it doesn't fit. So I guess I am polishing it out.
Better to find that now than when I have a big hunk of iron hanging off it.