I get too many requests errors during remote crawling stackexchange subdomains

I get too many requests errors during remote crawling stackexchange subdomains.
for example



I 2019/12/26 19:19:36 REJECTED https://unix.stackexchange.com/users/350165/user350165
Maybe you should extend throttling to subdomains?

Each Domain is handled separately and that includes all subdomains.

You can manually lower the PPM value in the Crawler Monitor.