YaCy-Bugtracker - YaCy
View Issue Details
0000656YaCy[All Projects] Generalpublic2016-06-08 23:042020-03-02 20:08
tybor 
 
normalblockalways
newopen 
none 
LinuxDebiantesting
YaCy 1.8 
 
0000656: Fails to start
I'm using version 1.83.9857 from http://debian.yacy.net [^]

I've been using yacy for a while, with my /var/lib/yacy/ being a sizeablt 165 gigabytes...

Today I discovered that yacy fails to start.

Inspecting logs it seems that there is some out-of-memory error, according to this:

W 2016/06/08 22:10:18 ConcurrentLog net.yacy.cora.util.SpaceExceededException: 3432600 bytes needed for RowCollection grow after OutOfMemoryError Java heap space: 36841872 free at Wed Jun 0
8 22:10:18 CEST 2016
net.yacy.cora.util.SpaceExceededException: 3432600 bytes needed for RowCollection grow after OutOfMemoryError Java heap space: 36841872 free at Wed Jun 08 22:10:18 CEST 2016
        at net.yacy.kelondro.index.RowCollection.ensureSize(RowCollection.java:276)
        at net.yacy.kelondro.index.RowCollection.addUnique(RowCollection.java:425)
        at net.yacy.kelondro.index.RowCollection.addUnique(RowCollection.java:403)
        at net.yacy.kelondro.index.RAMIndex.addUnique(RAMIndex.java:216)
        at net.yacy.kelondro.index.RAMIndexCluster.addUnique(RAMIndexCluster.java:133)
        at net.yacy.kelondro.index.RowHandleMap.<init>(RowHandleMap.java:103)
        at net.yacy.kelondro.blob.HeapReader.initIndexReadDump(HeapReader.java:179)
        at net.yacy.kelondro.blob.HeapReader.<init>(HeapReader.java:91)
        at net.yacy.kelondro.blob.HeapModifier.<init>(HeapModifier.java:58)
        at net.yacy.kelondro.blob.ArrayStack.<init>(ArrayStack.java:209)
        at net.yacy.kelondro.rwi.ReferenceContainerArray.<init>(ReferenceContainerArray.java:68)
        at net.yacy.kelondro.rwi.IndexCell.<init>(IndexCell.java:99)
        at net.yacy.search.index.Segment.connectRWI(Segment.java:159)
        at net.yacy.search.Switchboard.<init>(Switchboard.java:554)
        at net.yacy.yacy.startup(yacy.java:218)
        at net.yacy.yacy.main(yacy.java:726)
I 2016/06/08 22:10:18 HeapReader generating index for /usr/share/yacy/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160520172249804.blob, 6163 MB. Please wait.
W 2016/06/08 22:10:20 ConcurrentLog net.yacy.cora.util.SpaceExceededException: 455840 bytes needed for RowCollection grow after OutOfMemoryError Java heap space: 35845712 free at Wed Jun 08 22:10:20 CEST 2016
net.yacy.cora.util.SpaceExceededException: 455840 bytes needed for RowCollection grow after OutOfMemoryError Java heap space: 35845712 free at Wed Jun 08 22:10:20 CEST 2016
        at net.yacy.kelondro.index.RowCollection.ensureSize(RowCollection.java:276)
        at net.yacy.kelondro.index.RowCollection.addUnique(RowCollection.java:425)
        at net.yacy.kelondro.index.RowCollection.addUnique(RowCollection.java:403)
        at net.yacy.kelondro.index.RAMIndex.addUnique(RAMIndex.java:216)
        at net.yacy.kelondro.index.RAMIndexCluster.addUnique(RAMIndexCluster.java:133)
        at net.yacy.kelondro.index.RowHandleMap.putUnique(RowHandleMap.java:267)
        at net.yacy.kelondro.index.RowHandleMap$initDataConsumer.call(RowHandleMap.java:471)
        at net.yacy.kelondro.index.RowHandleMap$initDataConsumer.call(RowHandleMap.java:410)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
I 2016/06/08 22:10:20 ConcurrentLog shutdown of ConcurrentLog.Worker: injection of poison message
I 2016/06/08 22:10:20 ConcurrentLog terminating ConcurrentLog.Worker with 0 cached loglines.
I 2016/06/08 22:10:20 ConcurrentLog shutdown of ConcurrentLog.Worker: terminated
No tags attached.
Issue History
2016-06-08 23:04tyborNew Issue
2016-06-08 23:11tyborNote Added: 0001247
2020-03-02 19:12ktpluloNote Added: 0001505
2020-03-02 19:17ktpluloNote Edited: 0001505bug_revision_view_page.php?bugnote_id=1505#r470
2020-03-02 19:18ktpluloNote Edited: 0001505bug_revision_view_page.php?bugnote_id=1505#r471
2020-03-02 19:18ktpluloNote Edited: 0001505bug_revision_view_page.php?bugnote_id=1505#r472
2020-03-02 19:30ktpluloNote Edited: 0001505bug_revision_view_page.php?bugnote_id=1505#r473
2020-03-02 20:08ktpluloNote Edited: 0001505bug_revision_view_page.php?bugnote_id=1505#r474

Notes
(0001247)
tybor   
2016-06-08 23:11   
Somehow solved.
Changing "javastart_Xmx=Xmx600m" into "javastart_Xmx=Xmx2000m" and "javastart_Xms=Xms90m" into "javastart_Xms=Xms200m" made it start.
(0001505)
ktplulo   
2020-03-02 19:12   
(edited on: 2020-03-02 20:08)
I have this issue with 1.922/9964 now. The workaround above is very short-term, because YaCy breaks again soon. Hundreds of megabytes are enough for a few days during crawling.

INDEX: 25 G
INDEX/SEGMENTS: 24 G
HTCACHE: 7.2 G
HTCACHE/file.array: 6.8 G

Documents: 1.3 M
DHT words: 6.2 M

javastart_Xms=Xms130M

I had to change javastart_Xmm from Xmx1300m to Xmx1400m.