Just found this:
Grounded or Shorter Starter Field Coils: The original starter assembly uses internal field coils wrapped around the frame shell. If the insulation breaks down from age, heat, or a voltage conversion (such as running 12V through an old 6V starter), the field windings can short directly to the starter housing, causing sparks, smoke, or a dead short the moment power is applied.
Since this was a 12v conversion, it makes sense. Is it just an insulation thing?