If it's an AD truck, there should be a tension spring on the visor pivot. Not sure if it's adjustable or not though. My visor brackets are buried right now or I'd go out and look. Pull the visor bracket off the cab and check it out for adjustment capabilities.
That was it! It didn't occur to me that the spring tension needed to be really tightened down - a lot - actually, all the way to the stop. I learned this when I put the passenger side in today. Also, I found the arm on the passenger side had a nearly 45 degree tilt. Instead of it being 90 degrees straight across, it was pointing toward the floor on the opposite side of the truck. Something HAD to be done. I put the rod in the vice and pushed down. It took a few tries, but I got it bent to the right angle now, looks great! Did it to the driver's side too - even better! Now both sides are level, and since I tightened that screw all the way down, they hopefully will not move (unless I want them to).