Try disconnecting the lines, and look for a flare seat inside the cooler ports in the radiator. Some radiator-mounted coolers have the flare seats integral with the cooler, while others require adapters. If there is just an open hole into the cooler without a V-shaped area for the tubing flare to seat, it will be necessary to re-thread the cooler with a pipe thread tap and install a pipe-to-inverted flare adapter. Tightening the flare nut into the pipe thread will have damaged both fittings, so it might be necessary to cut the ends off the cooler lines, install new flare nuts, and re-flare the tubes.
I can't imagine a situation where flared tubing would still be able to move if the nut is tightened properly, so something is amiss and needs to be resolved before you fill the transmission and run the engine. It might be worth a trip to a radiator shop for an inspection of those fittings.
Jerry
