I replaced the return springs for both brake and clutch pedal, but that didn't change anything. I think I'll take it into the local brake shop and have the booster/check valve tested.
I really hate spending money I don't have, and I was hoping this would be something I could solve.