|Anonymous | Login | Signup for a new account||2019-09-15 12:05 CEST|
|Main | My View | View Issues | Change Log | Roadmap|
|View Issue Details|
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0000566||YaCy||[All Projects] General||public||2015-03-27 22:29||2016-09-28 15:11|
|Product Version||YaCy 1.8|
|Target Version||Fixed in Version|
|Summary||0000566: Still not shooting down|
|Description||Waiting now about 3 houres to shutdown using stopYacy.bat|
|Additional Information||Testet publisehed woraround worked 1 times.|
|Tags||No tags attached.|
|Attached Files|| yacy00.log [^] (954,911 bytes) 2015-05-03 14:56|
yacy00_(2015_ 06_27).log [^] (793,492 bytes) 2015-06-27 10:29
Screenshot from 2015-06-27 17:09:08.png [^] (3,081,699 bytes) 2015-06-27 10:30
YaCy_Starting.png [^] (672,473 bytes) 2015-06-28 06:36
YaCy_Shutdown.png [^] (165,676 bytes) 2015-06-28 20:27
yacy00_2015_07_03_Starting.log [^] (160,796 bytes) 2015-07-03 07:44
|I tooks about 1 day to start up! 15 GB Ran 6 cores 2,74 Mio Documents|
|now I'm at about 12 houres shutdown it still uses 17% CPU an 7.337.596 GB RAM|
edited on: 2015-05-03 15:00
I can confirm this issue.
YaCy version 1.83/9118.
Ubuntu 14.04 LTS, 24GB RAM (Allocating 18,000 MiB for YaCy), 4 core 8 thread CPU, 3TB HDD, Java 7u75, 1Gbps FTTH, language: Japanese.
YaCy is never shut down...
RAM used: 14.68 GB
RAM max: 17.2 GB
DISK used: (approx.) 395.61 GB
DISK free: 1,095.11 GB
The current size of the cache: 8126 MB for 305312 files, 27 KB / file in average
Maximum number of words in cache: 8000000
I attached a log file; yacy00.log.
When YaCy starts I notice a read burst on the disks lasting several minutes, with sequential accesses. This delays the boot until reads complete.
If the accesses are sequential, the disks sustain high transfer speed and YaCy starts quickly. You can check with `atop` if your accesses are sequential or random. The latter may be due to too many processes accessing the disk(s) concurrently, or filesystem fragmentation.
Thanks for the advice, Davide.
I tried 'atop'. But I do not understand the meaning of the results.
So, I upload the screenshot. (YaCy_Starting.png)
From your atop screenshots, is seems this time YaCy took a fair 30 minutes to start, after which your sda is reported as beginning to idle, indicating end of initial disk burst and start of cruise, regular service, which is a normal delay to me and apparently indicates a fairly large index, perhaps 500GB, given your disk performance.
You have 650% of wait time on the cpu, out of 800% (8 cores) so the start up strain is solely on the disk. The accesses are also sequential so nothing else appears to compete for disk IO, at least this time, during this set of atop samplings.
Based on the large amount of iops reported (about 800, given by 8000 divided by 10 seconds) you either have a hardware RAID (not visible from atop) or your read requests are pulled from the large 4GB RAM cache, speeding up accesses toward the disk. If the latter is the case and you don't have a hardware RAID, your RAM cache appears to be already populated with index data, supposedly read previously from sda; this would indicate that you have started YaCy more than once since the last computer reboot, and it is possible that the first start of YaCy was much slower than this, due to the cache being empty.
But from this set of samplings all seems OK.
|To be clear, large iops doesn't indicate that reads are served by the RAM cache, but still the 4GB cache may be playing a role here.|
I thank for your description, Davide!
As you said, I had started YaCy once before.(I have a single HDD)
I attached a new screenshot image file which is taken after issuing a command to shut down YaCy by pressing the GUI Shutdown button.
I attached a new log file that contains the following lines.
E 2015/07/03 14:23:29 HeapReader reclen == 0 at seek pos 11412961 in file /home/xxxx/yacy/DATA/HTCACHE/file.array/wB0BCU2HDv0Y.20150701175815495.blob
E 2015/07/03 14:23:35 HeapReader reclen == 0 at seek pos 26795858 in file /home/xxxx/yacy/DATA/HTCACHE/file.array/wB0BCU2HDv0Y.20150630170808531.blob
E 2015/07/03 14:24:32 HeapReader reclen == 0 at seek pos 20400906 in file /home/xxxx/yacy/DATA/HTCACHE/file.array/wB0BCU2HDv0Y.20150701190217890.blob
W 2015/07/03 14:28:20 HeapReader BLOB wB0BCU2HDv0Y.20150701180820484.blob: skiped not wellformed key \00\00\00\00\00\00\00\00\00\00\00\00 at seek pos 24778774
E 2015/07/03 14:29:48 HeapReader reclen == 0 at seek pos 40333151 in file /home/xxxx/yacy/DATA/HTCACHE/file.array/wB0BCU2HDv0Y.20150702031206918.blob
W 2015/07/03 14:29:58 Cache file and metadata size is not equal, starting a cleanup thread...
This file is recorded at the beginning of the YaCy.
|2015-03-27 22:29||Phrixos-IT||New Issue|
|2015-03-27 22:30||Phrixos-IT||Note Added: 0001025|
|2015-03-28 07:26||Phrixos-IT||Note Added: 0001026|
|2015-04-03 09:24||Collision||Note Added: 0001028|
|2015-04-03 09:41||Collision||Note Edited: 0001028||View Revisions|
|2015-04-04 04:45||Collision||Note Edited: 0001028||View Revisions|
|2015-04-04 16:15||BuBu||Relationship added||related to 0000505|
|2015-04-04 23:41||BuBu||Relationship added||related to 0000567|
|2015-05-03 14:56||Collision||File Added: yacy00.log|
|2015-05-03 15:00||Collision||Note Edited: 0001028||View Revisions|
|2015-06-27 10:29||Collision||File Added: yacy00_(2015_ 06_27).log|
|2015-06-27 10:30||Collision||File Added: Screenshot from 2015-06-27 17:09:08.png|
|2015-06-27 17:31||Davide||Note Added: 0001080|
|2015-06-28 06:36||Collision||File Added: YaCy_Starting.png|
|2015-06-28 06:43||Collision||Note Added: 0001081|
|2015-06-28 19:24||Davide||Note Added: 0001082|
|2015-06-28 19:29||Davide||Note Added: 0001083|
|2015-06-28 20:27||Collision||File Added: YaCy_Shutdown.png|
|2015-06-28 20:28||Collision||Note Added: 0001084|
|2015-07-03 07:44||Collision||File Added: yacy00_2015_07_03_Starting.log|
|2015-07-03 07:54||Collision||Note Added: 0001085|
|2016-09-28 15:11||BuBu||Relationship added||related to 0000689|
|Copyright © 2000 - 2019 MantisBT Team|