Dang, I had it replaced with master cylinder. A bad vacuum line to it wouldn't do that though so that's the only thing it could be.

Thank you