If you pull the manifolds apart and study on the flapper a bit, you will see it should stay in place. The exhaust will flow better out of the manifold due to the flapper valves design. It directs the flow around the corner and out the opening. Without the flap, the exhaust will still get out, but it will also bunch up under the intake. Does this hurt anything, probably not, but performance minded folks sure wouldn't want a restriction like that.

If you absolutely can't get it to move, leave it alone, it will not cause any problems with performance. My 250 has been running for years without a spring on the valve.
I suspect you might have a vacuum leak, they can cause all kinds of mysterious problems, lean being one of them.