Thanks for all the suggestions. It turns out I was able to use a 6" socket extension to grab the bushing and slot it back into place. I am just happy I didn't have to remove anything major, this project has been a fun little beast!