|Anonymous | Login | Signup for a new account||2020-12-02 04:25 CET|
|Main | My View | View Issues | Change Log | Roadmap|
|View Issue Details|
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0000745||YaCy||[All Projects] General||public||2017-06-01 02:55||2020-11-09 04:36|
|Product Version||YaCy 1.9|
|Target Version||Fixed in Version|
|Summary||0000745: Crawler speed reduces and Status log in GUI fails to refresh.|
|Description||Noticed the remote crawer had slowed down after 2 hours.|
Running 3 Yacys (Windows Version)in 1 playonlinux drive XP in use.
The Top VIRT memory is 45 % reduction in the figures when compaired to just running on Ubuntu 16.04.
I am downloading SP3 for playonlinux.
|Additional Information||I wrote a program to extract errors and errors and warnings from the log files.|
There is a 8 yacy log files.
|Tags||No tags attached.|
|Attached Files|| logfiles.tar.gz [^] (1,309,594 bytes) 2017-06-01 02:55|
Threads.txt [^] (1,911 bytes) 2017-06-05 23:51 [Show Content]
Thank you smokingwheels for sharing a part of your threads dump. For this case this may be more helpful than the logs (for now, I did not found any clue about the issue by reading them). Can you share the full threads dump?
By the way, the part you shared may suggest there is a problem with file handlers. I already started cleaning up potential leaks in code dealing with FileInputStreams (https://github.com/yacy/yacy_search_server/commit/d98c04853dd39a452b0b98fb269a2013396cbb42 [^]). But there are probably other code paths to improve, notably the ones using RandomAccessFiles.
edited on: 2017-06-06 14:53
I think there is a problem when doing a shutdown while crawling, it leaves the crawler Que's open on disk or something.
Clone and Run as root doing a crawl then shutdown Yacy.
I used chmod -R 755 /usr/yacy... then as a user moved Yacy's files to different place it still crawls when started but the 0003.stacks and folders have errors with cut and past they are all left behind.
Possible Fix is to create a crawler stacks index counter file separately thus retain compatibility with earlier versions, for times someone uses pkill java to terminate it.
edited on: 2020-11-09 04:36
that's a good idea
|2017-06-01 02:55||smokingwheels||New Issue|
|2017-06-01 02:55||smokingwheels||File Added: logfiles.tar.gz|
|2017-06-05 23:51||smokingwheels||File Added: Threads.txt|
|2017-06-06 06:56||luc||Note Added: 0001407|
|2017-06-06 14:53||smokingwheels||Note Added: 0001408|
|2017-06-06 14:53||smokingwheels||Note Edited: 0001408||View Revisions|
|2020-11-09 04:35||bobpaul||Note Added: 0001509|
|2020-11-09 04:35||bobpaul||Note Edited: 0001509||View Revisions|
|2020-11-09 04:36||bobpaul||Note Edited: 0001509||View Revisions|
|Copyright © 2000 - 2020 MantisBT Team|