Hey everyone. I have a 67 gmc c10 with a 327 engine and 4 speed manual transmission I recently started fixing up. I can’t seem to figure out why the gears keep grinding every time I downshift. It shifts up perfectly. Downshifting from 3 to 4 works ok and gets progressively worse as you go down. I end up having to stop before shifting up again. Does this mean the clutch or clutch brake is going out? Would the whole engine and trans need to come out for a fix? Any help is appreciated.

Thanks

1954 Chevy 3800