chevy52,
If this is the stomp starter, there could be worn linkage issues.
When installing a new starter, does the old linkage get reused!
If the starter switch is actuated before the drive gear is engaged, it will grind.
A worn ring gear could cause this, but checking the linkage would be a good starting point.
When you say you 'swapped out' the starter, does this mean you installed a new or rebuilt starter.
Stuart