I blew up a muffler once in a '62 Impala. Coil went bad so no ignition, leading to overload of fuel. Then BAM!!!! Replaced coil & muffler and all was fixed. Also same problem in '82 C20, but mufflers survived backfire.
So I would say probably coil. Maybe points, but definetly an ignition problem. Your exhaust should be more than big enough for a 6cyl & fine for a V8.