Use low speed for the coarse polishing steps, as 180 grit generates a lot of heat and puts deep scratches in the metal if it's run too fast or you use too much pressure on the wheel. As the grit gets finer and the surface gets smoother kick the speed up and gently remove the scratches the previous compound left. When the surface won't get any better-looking, switch to a finer grit and remove the scratches that remain. My hard felt wheel is 8" diameter and I run a slightly faster surface speed than with the muslin wheels. That final polish is done with a VERY light touch, and keep holding the part at different angles so you don't leave a directional final finish.
Jerry