Sure you can but there are all kinds of reasons that is a bad idea. Including the one you mentioned.
1. Motor and exhaust gets real wet and dirty. Can cause motor to die in a puddle.
2. Air flow is greatly disturbed and motor can over heat.
3. Structural movement and noise.
4. Components mounted to inner fender and wiring harness must be moved.
5. Radiator cross member and deflectors sometimes attach to inner fenders.
6. Hood fit
7. Air cleaner intakes more dirt
8. Ugly
Thanks for the feedback- I have the passenger side one, but have been having some trouble determining what everything bolts to, so I thought maybe I could just leave them out. Sometimes the FAM is a bit hard to follow.