It is always better to remove everything. The only question is will you get frustrated, tired, and lose interest if you can't finish the rebuild in a short time. There are plenty of partially finished projects out there littering the hotrod landscape.
