Sideways play in a stovebolt distributor is a chronic problem, but there is a fix. Remove the grease cup assembly that is threaded into the housing just above the vacuum advance. There is a spring and a plastic disc under the base of the lubricator that is supposed to put spring pressure against the shaft. Grease can solidify and keep the spring from putting a side load onto the shaft, and/or the plastic disc can wear out. A piece of phenolic plastic 5/16" in diameter and about 1/4" long can be used to replace the pressure disc. The phenolic plastic is available on Ebay- - - -a piece 3 or 4 inches long can fix a dozen distributors.
I fit a caged Torrington needle bearing to the top of the distributor housing and press a hardened steel sleeve onto the shaft, totally eliminating shaft wobble. Don't discard a wobbly distributor- - - -they're very easy to fix!
Jerry