So, I’m trying to index ClearNet websites as well as alternative network sites (Onion Sites, I2P sites, LokiNet sites, Yggdrasil sites)
Clearnet, Yggdrasil Network is working as intended.
LokiNet is seen internally as a local address, same as Tor onion sites. I haven’t tried I2P sites yet but that shouldn’t be a problem I don’t think.
When I set, “example.loki” as the index point, all seems good until the next page where I see the error:
Crawling of "http://example.loki " failed. Reason: denied_(the host ‘example.loki’ is local, but local addresses are not accepted: 172.16.0.5)/
Anyway to crawl these networks and have the results combined in one spot?