Check the top of the distributor shaft to see if you can shake it side to side. It sounds like the point gap/dwell is changing due to distributor shaft wobble from a worn upper bushing, which is changing the timing. For every degree of dwell change, the timing changes two degrees. At higher speed, the timing will appear to stabilize.
Jerry