Agree with JW51 and Gdads51.
There's nothing to keep the drag link joints exactly lined up. They're going to go to the point where the edge of the socket prevents them from moving any more. Their function is to maintain the force along the length of the drag link, and as long as that isn't loose, then it's not going to be the cause of wandering. If you were to install a new drag link, it would likely move to that position as well.
A more common cause of wandering is likely incorrect caster, tie-rod, or wheel bearing looseness. I would check all the steering components and check the wheel alignment.
