Yes, that's what I've been saying repeatedly. If you reinstall it without changing that piece, you'll just be putting the problem right back in there. The starter drive has a one-way clutch built into it to avoid overspeeding the starter armature when the engine fires up. Once the clutch begins to fail, it can freewheel both ways occasionally, and the starter shaft spins without turning the flywheel. The clutch will catch occasionally, and work right for a time before it fails completely.
Jerry