Anonymous | Login | Signup for a new account | 2021-04-16 05:19 CEST | ![]() |
Main | My View | View Issues | Change Log | Roadmap |
View Issue Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||||
ID | Project | Category | View Status | Date Submitted | Last Update | ||||
0000758 | YaCy | [All Projects] General | public | 2017-06-14 15:20 | 2017-06-15 23:03 | ||||
Reporter | Collision | ||||||||
Assigned To | administrator | ||||||||
Priority | immediate | Severity | major | Reproducibility | always | ||||
Status | resolved | Resolution | fixed | ||||||
ETA | none | ||||||||
Platform | Intel Core-i7 | OS | Ubuntu | OS Version | 16.04 LTS | ||||
Product Version | YaCy 1.9 | ||||||||
Target Version | Fixed in Version | ||||||||
Summary | 0000758: Cpu load is always 100%. | ||||||||
Description | After I updated YaCy to version 1.921/9236, a problem occurred. The CPU is always busy since YaCy started. | ||||||||
Steps To Reproduce | Start up YaCy 1.921/9236. | ||||||||
Additional Information | I upload a Visual VM thread dump file and a screenshot. | ||||||||
Tags | No tags attached. | ||||||||
Attached Files | ![]() ![]() | ||||||||
![]() |
|
(0001433) Collision (reporter) 2017-06-14 15:32 |
By the way, today the autocratic regime legislates the law which punishes collusion and the democracy of Japan is about to be destroyed..... |
(0001434) luc (reporter) 2017-06-15 07:24 |
Your dump would suggest the load is mainly on Solr. Maybe the upgrade from Solr 5.5 to 6.6 is in cause... Were you performing some search requests, crawls or any other operations on your peer while producing the thread dump? Or is your CPU always at 100% even when your peer is doing nothing else than waiting and eventually only answering to other peers? |
(0001435) Collision (reporter) 2017-06-15 08:48 |
Just by starting up YaCy, I have not done any operation. The overload condition will continue until YaCy is forcibly terminated. In addition,the storage access is also busy for a long time after startup. |
(0001436) Collision (reporter) 2017-06-15 08:54 |
It seems I can not access 'crawler monitor' and 'index browser'. |
(0001437) luc (reporter) 2017-06-15 12:30 |
The crawler monitor page also takes more than one minute to display on a small test peer of mine. Using the "Sampler" feature of JVisualVM reveals that much CPU is used in the "org.apache.solr.handler.component.SuggestComponent$SuggesterListener.buildSuggesterIndex()". Looks like other people experienced this using Solr: see https://issues.apache.org/jira/browse/SOLR-6845 [^] Can you check if modifying your YaCy defaults/solr/solrconfig.xml by adding "<str name="buildOnStartup">false</str>" to the "<searchComponent name="suggest" class="solr.SuggestComponent">" configuration solve your problem? It looks like it works finer for me after that modification. |
(0001438) Collision (reporter) 2017-06-15 13:52 |
Yes! You are right, luc! I tried as you said, the problem was solved. Thank you very much for your kind reply. By the way, does this problem affect other users? |
(0001439) luc (reporter) 2017-06-15 14:03 |
Yes other users that applied the upgrade are potentially affected, but I guess the problem is more or less visible depending on the index size and the processor you have. How many documents do you have in your index? By the way, thanks to your quick feedback, I am more sure of the relevency of the proposed fix and I will commit on GitHub. |
(0001440) Collision (reporter) 2017-06-15 15:15 edited on: 2017-06-15 15:21 |
I have 34,665,660 documents and 26,569,252 RWIs in my YaCy index. YaCy is using 680.47 GB of HDD space. 'yacy/DATA/INDEX/freeworld/SEGMENTS/solr_6_6' contains 793 items, 554.3 GB used. For reference, My PC has Intel Core-i7 3770T 4 core CPU, 32GB DDR3-SDRAM, 3TB HDD, 1Gbps FTTH network. Thank you for your work! |
(0001441) Collision (reporter) 2017-06-15 17:56 |
One more infomation: 1,367,747,785 webgraph Edges I have. |
(0001444) BuBu (developer) 2017-06-15 23:03 |
see commit by @luc https://github.com/yacy/yacy_search_server/commit/f6e8d7171865087d518f17278ad35a082b6cbfff [^] |
![]() |
|||
Date Modified | Username | Field | Change |
2017-06-14 15:20 | Collision | New Issue | |
2017-06-14 15:20 | Collision | File Added: Screenshot from 2017-06-14 21-53-07.png | |
2017-06-14 15:21 | Collision | File Added: threaddump-1497444704828.tdump | |
2017-06-14 15:32 | Collision | Note Added: 0001433 | |
2017-06-15 07:24 | luc | Note Added: 0001434 | |
2017-06-15 08:48 | Collision | Note Added: 0001435 | |
2017-06-15 08:54 | Collision | Note Added: 0001436 | |
2017-06-15 12:30 | luc | Note Added: 0001437 | |
2017-06-15 13:52 | Collision | Note Added: 0001438 | |
2017-06-15 14:03 | luc | Note Added: 0001439 | |
2017-06-15 15:15 | Collision | Note Added: 0001440 | |
2017-06-15 15:16 | Collision | Note Edited: 0001440 | View Revisions |
2017-06-15 15:21 | Collision | Note Edited: 0001440 | View Revisions |
2017-06-15 17:56 | Collision | Note Added: 0001441 | |
2017-06-15 23:03 | BuBu | Note Added: 0001444 | |
2017-06-15 23:03 | BuBu | Status | new => resolved |
2017-06-15 23:03 | BuBu | Resolution | open => fixed |
2017-06-15 23:03 | BuBu | Assigned To | => administrator |
Copyright © 2000 - 2021 MantisBT Team |