Use two or three 1/8" pop rivets. You won't be able to see them and surely you can get the rubber past that little bit.....
In fact, my piece won't be here till monday and the painter has already painted the inside of my truck. We plan to pop rivet the pice to the inner panel.