I don’t think that’s a direct sign of trouble, at least not in and of itself.
It’s just a ball and socket type of thing that wanders wherever it wants to go. Might take a pry bar to various points and see if you can detect any play or looseness. That would be a bigger concern that how it wants to orient around the ball.
