Everything you've heard so far is correct, and from the sound of it your problem is either the return spring in the push-button on the top of the starter motor housing or the pedal linkage spring located at the starter motor.
There is one more spring, and that is the internal one in the starter motor that returns the armature and internal starter drive linkage when you take your foot off the pedal. All three springs are necessary to return the armature, puch button and foot pedal linkage to rest.
If you can't fix it with one of the first two springs (or by cleaning and lubricating a little on the foot pedal linkage or the push-button contactor), then look internally on the starter motor.
As you are working on the "hot" electrical stuff, be sure to disconnect the negative battery terminal.
