Sounds like Paul could block that 57.141.4.xx IP address range and take care of the issue.
Yes, it's getting downright irritating. We get twice as much traffic from crawlers as we do from our own members, and it consistently degrades the performance when they show up. I'm already rate-limiting Googlebots, and i've blocked the googleusercontent networks completely. Looks like I'll also have to block the meta crawlers.
The least these jerks could do, if they want to feed their AI, is ask our permission.
