HUMMMM!!!!! Sounds as thow you have hooked up your push button to the wrong terminal on the headlamp switch. If it were mine and I didn't want stomp start. I would put on a 12v starter and solenoid, then install an ignition switch with a start circuit.
But then again if I had a stomp starter I would leave it alone cause they are so cool.

I didn't do it, no one saw me do it, you can't prove anything! "Bart Simpson"