if you mean replacing the felt run channel top and back, no you don't really need to remove the windows, although if you're re-felting the division bar channel [glue in felt cloth for front of main window], you will only be able to do the whole thing if the division bar is out .... the
beaded window seals at the bottom of the opening are usually called 'catwhiskers' and yes, they're felt - there's also
a rubber piece that the glass mounts in [fits into the bottom sash] that has a lip that butts against the outside of the opening when the window is up
Bill