Adjusting the brakes until you "heard just a bit of rubbing" might be your problem. Here's what I do:
Run each adjuster out until the wheel gets very hard to turn (this centers the shoes to the drum).
Then back them off until theres a slight rubbing noise.
Try that and see if that makes a difference. Good luck!!
