So, on Sunday I replaced the throwout bearing. Now it is as quiet as can be. When I first installed it, it made some noise when the clutch pedal was depressed but it soon quieted down. Seems like that was the fix. I also replaced both u-joints while I had the driveshaft out.