It doesn't take anything funky inside the distributor for that kind of thing to happen. Back when I was a kid driving a Model A, a favorite trick was turning the ignition off momentarily and back on to produce a pop out of the exhaust. Depowering the coil at a random point will fire a spark plug when it's not supposed to be (like when an exhaust valve is open). I'm with Jerry that the issue is with either the ignition switch or the wiring to the coil.
