Sounding more and more like you have a blockage in the water passages inside the engine. The waterpump cannot pump past the blockage so pressure builds until the cap cannot hold it back any longer. Out the overflow it comes.
I'd say more like blocked radiator passages. The water pump sucks water from the bottom of the radiator, and then pushes it through the block and into the top of the radiator. If the radiator tubes are blocked, the water will have nowhere to go but the overflow when the thermostat opens.
[on edit] went back and saw that you have a new radiator. But the pic doesn't look that "new". Also just because the cap attaches at the top, doesn't mean it seals the section below the overflow tube. If you have a Chrysler radiator, then you need the appropriate Chrysler cap.
