YaCy-Bugtracker - YaCy
View Issue Details
0000730YaCyWishlist - Wunschlistepublic2017-03-28 19:342020-12-16 06:24
smokingwheels 
 
normalminoralways
newopen 
none 
X86Linux Debian +Ubuntu
YaCy 1.9 
 
0000730: Some web sites have URL's with ;amp;amp. hundreds of them.
When crawling some sites, I have noticed a few sites having a URL's that suffice/fix with www.domain.com/somepage.html;amp;amp;amp so on for at least 3 to 4 lines in the crawler monitor.

If I find then now I terminate the crawl or blacklist the site because it just slows down my slow PC.

Maybe an option to bypass the sites if one wishes?


 
No tags attached.
Issue History
2017-03-28 19:34smokingwheelsNew Issue
2020-12-16 06:23hnnananNote Added: 0001517
2020-12-16 06:24hnnananNote Edited: 0001517bug_revision_view_page.php?bugnote_id=1517#r487

Notes
(0001517)
hnnanan   
2020-12-16 06:23   
(edited on: 2020-12-16 06:24)
Thank you all for your kindness and wish you happiness
Official website:https://www.assortlist.com [^]