YaCy-Bugtracker
Main | My View | View Issues | Change Log | Roadmap
  

View Issue Details [ Jump to Notes ] [ Issue History ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000745YaCy[All Projects] Generalpublic2017-06-01 02:552020-11-09 04:36
Reportersmokingwheels 
Assigned To 
PrioritynormalSeverityminorReproducibilityrandom
StatusnewResolutionopen 
ETAnone 
PlatformOSOS Version
Product VersionYaCy 1.9 
Target VersionFixed in Version 
Summary0000745: Crawler speed reduces and Status log in GUI fails to refresh.
DescriptionNoticed the remote crawer had slowed down after 2 hours.
Running 3 Yacys (Windows Version)in 1 playonlinux drive XP in use.
The Top VIRT memory is 45 % reduction in the figures when compaired to just running on Ubuntu 16.04.
I am downloading SP3 for playonlinux.


Additional InformationI wrote a program to extract errors and errors and warnings from the log files.
There is a 8 yacy log files.
TagsNo tags attached.
Attached Filesgz file icon logfiles.tar.gz [^] (1,309,594 bytes) 2017-06-01 02:55
txt file icon Threads.txt [^] (1,911 bytes) 2017-06-05 23:51 [Show Content]

- Relationships

-  Notes
(0001407)
luc (reporter)
2017-06-06 06:56

 Thank you smokingwheels for sharing a part of your threads dump. For this case this may be more helpful than the logs (for now, I did not found any clue about the issue by reading them). Can you share the full threads dump?

By the way, the part you shared may suggest there is a problem with file handlers. I already started cleaning up potential leaks in code dealing with FileInputStreams (https://github.com/yacy/yacy_search_server/commit/d98c04853dd39a452b0b98fb269a2013396cbb42 [^]). But there are probably other code paths to improve, notably the ones using RandomAccessFiles.
(0001408)
smokingwheels (reporter)
2017-06-06 14:53
edited on: 2017-06-06 14:53

 Ok Thanks.
I think there is a problem when doing a shutdown while crawling, it leaves the crawler Que's open on disk or something.

Clone and Run as root doing a crawl then shutdown Yacy.

I used chmod -R 755 /usr/yacy... then as a user moved Yacy's files to different place it still crawls when started but the 0003.stacks and folders have errors with cut and past they are all left behind.

Possible Fix is to create a crawler stacks index counter file separately thus retain compatibility with earlier versions, for times someone uses pkill java to terminate it.
(0001509)
bobpaul (reporter)
2020-11-09 04:35
edited on: 2020-11-09 04:36

 that's a good idea
https://www.netbestgroup.com [^]

- Issue History
Date Modified Username Field Change
2017-06-01 02:55 smokingwheels New Issue
2017-06-01 02:55 smokingwheels File Added: logfiles.tar.gz
2017-06-05 23:51 smokingwheels File Added: Threads.txt
2017-06-06 06:56 luc Note Added: 0001407
2017-06-06 14:53 smokingwheels Note Added: 0001408
2017-06-06 14:53 smokingwheels Note Edited: 0001408 View Revisions
2020-11-09 04:35 bobpaul Note Added: 0001509
2020-11-09 04:35 bobpaul Note Edited: 0001509 View Revisions
2020-11-09 04:36 bobpaul Note Edited: 0001509 View Revisions

Copyright © 2000 - 2021 MantisBT Team
Powered by Mantis Bugtracker