The problem seems to lie in the design of the latch mechanism itself.
If you study the mechanism you'll see that the way they are designed puts two flat moving surfaces in contact with each other. All of this contact surface increases friction which overcomes the effect of the spring keeping the handle from returning to it's horizontal position. To compound the problem, the spring appears to of marginal strength lacking adequate force to return the handle to the horizontal under all but the most ideal conditions.
You can buy new springs which just may fix the problem if you have older mechanisms that are worn loose. I have new ones so I've mixed some jewelers rouge with light oil in an attempt to polish the contact surfaces and reduce the friction, but there hasn't been enough time to determine if this helps.
Bottom line... live with the sag, it the nature of the beast.
