Jgalt- Welcome Sedan-er. My stovebolt is the same as yours- '54 Bel-Air 4 door with 235 and three speed stick.
You can get the pan off without moving engine. However you have to unbolt the steering center link, and getting to the 3/4" bolt in the x member is somewhat difficult. Drop steering center link, unbolt pan and carefully finagle it out from around front x member and oil pump pickup.
But if the washer is laying flat on the pan floor, it shouldn't go anywhere. the trick is- are you sure its not in the way of anything?
If I may ask, where did the washer come from? I've never seen a washer as you describe on a 235. And believe me, I've looked. For about 6 months.
good luck
chip