My guess is the end of the speedometer or the cable end, were the cable rides, is worn just enough to not spin the head. If the cable was longer it would engage deeper into the speedometer, which I suspect is happening when using the drill motor. Is the end of the cable still squared off? A new inner cable would likely solve your problem. Do you have a small ball bearing you can drop into the adapter end so the cable doesn't go so deep? This would put more cable at the speedometer head, just be sure the driven end is a blind hole so the ball bearing stays where you put it.

Joe

1937 Chevrolet Inline-powered Streetrod