Most cutout relays have two adjustments, and yes, they're usually made by bending spring stops. The first adjustment is "closing voltage" and it's made by either observing the voltage produced at the Arm. terminal of the generator as the idle speed is raised, or by using a variable resistor between the FLD. terminal and ground with the F wire disconnected at the regulator. I use a 0-100 ohm 200 watt variable resistor to control field current while making this adjustment. It's much more accurate than tinkering with the idle speed. Slowly decrease the field resistance while watching the armature voltage. When the armature voltage gets approximately 1/2 volt above battery terminal voltage the cutout relay points should close and remain closed as the generator charges the battery. Start dropping the armature voltage by increasing the field resistance and as the voltage drops low enough to allow the battery to try to discharge through the generator the cutout should open. A "reverse current" of about 1-2 amps should be sufficient to cause the relay to spring open.
Here's where it gets tricky- - - -if the movable point is pulled down tightly against the cutout relay coil, sometimes the points won't open quickly enough. There's an "air gap" adjustment that must be made that allows the points to open and close properly, and I'll bet yours is too wide with the points open. You're having to weaken the spring tension too much to get the points to close, and there's not enough spring to pop the points open before the reverse current takes over and re-magnetizes the coil in the opposite direction. Reduce the air gap slightly between the movable point and the coil, and you'll be able to increase the spring tension enough to make the opening reliable with the right amount of reverse current.
IT IS ABSOLUTELY ESSENTIAL TO HAVE AN ACCURATE AMMETER IN THE CHARGING CIRCUIT IN SERIES WITH THE "BAT" WIRE WHILE MAKING THESE ADJUSTMENTS! Trying to do it with a voltmeter alone is impossible. So is trying to use the dash ammeter- - - -not accurate enough. It's also a very good idea to have analog volt and amp meters (with a sweep needle, not digital) while adjusting a voltage regulator, as the vibrating points create enough "noise" to make a digital readout go bonkers.
The above procedures are ones I learned well over 50 years ago. They worked then, and they still do now!
Jerry