Several possibilities. One is the check valve for the fuel inlet. I have one here that the fuel enters through the center of the accelerator pump. The problem is that it's installed backwards and permanently pressed together, so there's no way to remedy the issue. If you apply vacuum to the port, it closes off, and if you apply pressure, it allows it to pass to the back side of the cup. Another possibility is that the plunger spring is too light, too weak, or too short. The plunger spring does the actual work, while the linkage returns it to the closed throttle position. You CAN add washers between the spring and the plunger cup to increase tension or decrease the release distance of the linkage.
You didn't say what happens if you move the throttle while looking down the carb. Does it begin to squirt fuel immediately, or does the throttle plate begin to open before fuel begins to flow? If there is a lag, then add washers to the spring or check the linkage to make sure that the hook that holds the pump plunger begins to drop as soon as the throttle moves. Without having one right in front of me. It's hard to say, because I don't know where the fuel inlet for the pump is. I know I have one around here somewhere, but it's not at my computer!
Lastly, if you have your idle set to about 400 RPM like I used to do on my '50 (just because I can and I love the sound of it!) Then you idle is set too low, OR your idle mixture is a bit too lean. There are transfer ports that utilize the idle circuit passages. They're the long slots just above the throttle plate. They provide additional fuel to make the transition to the primary metering system in the band between idle and the amount of air flow needed to pull fuel through the primary.
