Joe - likely you got out the only set screw, but sometimes the shafts and/or handle can get chewed up from the set screw being loose and the handle being abused, you may have to use a lot of muscle to get it off and you'll likely find the shaft is too chewed up to be useable
Bill