My guess would be the starter switch on the top of the starter. If the contacts are burnt at all, mashing it all the way down may not make good contact, but letting off may get better contact. I would inspect the contact on the switch itself and see what you find. You can get your battery load tested, but if the starter turns normally once you do your dance with the starter pedal, I wouldn't suspect the battery.
