The drive may be slipping. Many times they sound like they are grinding against the flywheel teeth when they are slipping inside themselves. Either way, check the drive carefully, and check the teeth on the fly wheel while the starter is off to make sure it isn't messing up the teeth.
