The button goes into the contact assembly as a press fit, most likely it is just rusted. Use some nylon prying tools and work it side-to-side. https://www.flickr.com/photos/advance-design_parts_co/46694747735/in/album-72157631872034068/
This page is from the FAM (Factory Assembly Manual 1947-54 (mostly 54), available from most vendors.