Because a 6 volt system operates on half the voltage it uses twice the amperage to run the starter.
I have seen a number of starters that acted as yours that only needed all the brush connections tightened.
All connections need to be as clean and tight as you can get them on a 6 volt system.
