Thats a part people usually overlook if the drums arent scored. I usually have mine turned or at least checked for roundness when doing a brake job.
Like you said,,,,,a difference like day and night.
Marty