Mine used to do that too. The button on the starter would stick down and the starter would not shut off. You might need to replace the contact switch on the top of the starter. Easy to do. Mine was new, so I fixed it with a generous application of silicone spray. It works fine now.
George