As you say, priming the carb does not make the pump operate better. In a way it does, because of what Mike B. said. The extra time the pump needs to get fuel into the carb bowl is obtained by you prime starting the motor and it running separate from the pump. The pump "catches up" but it seems like the carb prime had an effect on pump pressure. Letting pump run longer before starting will prove this theorem.
FI motors have this pressure-up feature built-in.
You may also have a weak pump due to age or blockage. Probably time to replace. Make sure it is a 3.5-4 max psi pump.

Even's test is the next thing to do. You can even time it. Do this test after it has been sitting.

The only thing I can think of for why it does it only sometimes is the weather. I.E. evaporation rate of bowl gas.

It is slightly possible you have a intermittent venting problem. This could be weather related also. I.E. barometric pressure.

How did I get on this weather kick? I don't know where all that is coming from? The weather I guess.

Right is right, even if everyone is against it, and wrong is wrong, even if everyone is for it. - William Penn