View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000527YaCyWishlist - Wunschlistepublic2015-01-25 16:142021-05-18 08:08
Assigned To 
PlatformOSOS Version
Product VersionYaCy 1.8 
Target VersionFixed in Version 
Summary0000527: Wishlish: Different Proxy Settings for Crawling and YaCy<->Yacy
DescriptionI would like to be able to use no proxy for YaCy<->Yacy communications but do use a http proxy for http and https connections to websites being crawled.

A huge amount of websites have, for some reason, hired Cloudflare to deny their users service. It is unclear exactly why a large number of website owners decide to deny a large portion of their users service but they clearly do.

I would like to be able to crawl with a proxy that goes through Tor and still do YaCy<->YaCy communications over non-Tor. The result of this is that all the sites blocked by Cloudflare would not be crawled and therefore not show up in the search results.

I do not mind that they can see that I am running a YaCy node and them tracking the search entries is not that bad. YaCy really should start encrypting all yacy<->yacy communications, seriously, why the flying fuck isn't it doing this - it would be so each for each yacy node to generate a keypair and use this for yacy<->yacy but that is a another bug.

I really do like to be able to visit the search results I get from YaCy over Tor. A bad result of doing crawls over the clearnet and visiting search results over Tor is that some sites do not get blocked by Cloudflare over clearnet and are included in the search results but they don't work over Tor. Thus it would be better to do the crawls over Tor but this aparently also makes yacy<->yacy go over Tor and that is not required. Thus one proxy setting for yacy<->yacy and another for crawling would be great.
TagsLol, yacysearch
Attached Files

- Relationships

-  Notes
sceneryeor (reporter)
2021-05-18 08:08

Others laughed at me as crazy, I laughed at others and couldn't see through.
https://www.firstwritehome.com [^]

- Issue History
Date Modified Username Field Change
2015-01-25 16:14 oyvinds New Issue
2019-07-28 08:43 Nopolop Tag Attached: Lol
2019-07-28 08:45 Nopolop Tag Attached: yacysearch
2021-05-18 08:08 sceneryeor Note Added: 0001555

Copyright © 2000 - 2021 MantisBT Team
Powered by Mantis Bugtracker