Garry, It could be that ball hasn't been separated for quite some time, if ever??? You did remove the 4 bolts holding the ball retaining collar on like Kevinski said, yes? If this is the case, and as you say you've already unthreaded the rear collar nut, you are going to have to try a little gentle persuasion to break it free. I would try to grasp the narrow collar tube and try slight side to side and up and down motion to see if it will loosen up the ball and let it slide back.
Get back and let us know if that worked or not. BTW - If you can take and post a couple of pictures, it would help folks see what things look like and possible point out something being overlooked.
