0000656: Fails to start - YaCy-Bugtracker

View Issue Details [ Jump to Notes ]

[ Issue History ] [ Print ]

Project

Category

View Status

Date Submitted

Last Update

0000656

YaCy

[All Projects] General

public

2016-06-08 23:04

2020-03-02 20:08

Reporter

tybor

Assigned To

Priority

normal

Severity

block

Reproducibility

always

Status

new

Resolution

open

ETA

none

Platform

Linux

Debian

OS Version

testing

Product Version

YaCy 1.8

Target Version

Fixed in Version

Summary

0000656: Fails to start

Description

I'm using version 1.83.9857 from http://debian.yacy.net [^]

I've been using yacy for a while, with my /var/lib/yacy/ being a sizeablt 165 gigabytes...

Today I discovered that yacy fails to start.

Inspecting logs it seems that there is some out-of-memory error, according to this:

W 2016/06/08 22:10:18 ConcurrentLog net.yacy.cora.util.SpaceExceededException: 3432600 bytes needed for RowCollection grow after OutOfMemoryError Java heap space: 36841872 free at Wed Jun 0
8 22:10:18 CEST 2016
net.yacy.cora.util.SpaceExceededException: 3432600 bytes needed for RowCollection grow after OutOfMemoryError Java heap space: 36841872 free at Wed Jun 08 22:10:18 CEST 2016
        at net.yacy.kelondro.index.RowCollection.ensureSize(RowCollection.java:276)
        at net.yacy.kelondro.index.RowCollection.addUnique(RowCollection.java:425)
        at net.yacy.kelondro.index.RowCollection.addUnique(RowCollection.java:403)
        at net.yacy.kelondro.index.RAMIndex.addUnique(RAMIndex.java:216)
        at net.yacy.kelondro.index.RAMIndexCluster.addUnique(RAMIndexCluster.java:133)
        at net.yacy.kelondro.index.RowHandleMap.<init>(RowHandleMap.java:103)
        at net.yacy.kelondro.blob.HeapReader.initIndexReadDump(HeapReader.java:179)
        at net.yacy.kelondro.blob.HeapReader.<init>(HeapReader.java:91)
        at net.yacy.kelondro.blob.HeapModifier.<init>(HeapModifier.java:58)
        at net.yacy.kelondro.blob.ArrayStack.<init>(ArrayStack.java:209)
        at net.yacy.kelondro.rwi.ReferenceContainerArray.<init>(ReferenceContainerArray.java:68)
        at net.yacy.kelondro.rwi.IndexCell.<init>(IndexCell.java:99)
        at net.yacy.search.index.Segment.connectRWI(Segment.java:159)
        at net.yacy.search.Switchboard.<init>(Switchboard.java:554)
        at net.yacy.yacy.startup(yacy.java:218)
        at net.yacy.yacy.main(yacy.java:726)
I 2016/06/08 22:10:18 HeapReader generating index for /usr/share/yacy/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160520172249804.blob, 6163 MB. Please wait.
W 2016/06/08 22:10:20 ConcurrentLog net.yacy.cora.util.SpaceExceededException: 455840 bytes needed for RowCollection grow after OutOfMemoryError Java heap space: 35845712 free at Wed Jun 08 22:10:20 CEST 2016
net.yacy.cora.util.SpaceExceededException: 455840 bytes needed for RowCollection grow after OutOfMemoryError Java heap space: 35845712 free at Wed Jun 08 22:10:20 CEST 2016
        at net.yacy.kelondro.index.RowCollection.ensureSize(RowCollection.java:276)
        at net.yacy.kelondro.index.RowCollection.addUnique(RowCollection.java:425)
        at net.yacy.kelondro.index.RowCollection.addUnique(RowCollection.java:403)
        at net.yacy.kelondro.index.RAMIndex.addUnique(RAMIndex.java:216)
        at net.yacy.kelondro.index.RAMIndexCluster.addUnique(RAMIndexCluster.java:133)
        at net.yacy.kelondro.index.RowHandleMap.putUnique(RowHandleMap.java:267)
        at net.yacy.kelondro.index.RowHandleMap$initDataConsumer.call(RowHandleMap.java:471)
        at net.yacy.kelondro.index.RowHandleMap$initDataConsumer.call(RowHandleMap.java:410)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
I 2016/06/08 22:10:20 ConcurrentLog shutdown of ConcurrentLog.Worker: injection of poison message
I 2016/06/08 22:10:20 ConcurrentLog terminating ConcurrentLog.Worker with 0 cached loglines.
I 2016/06/08 22:10:20 ConcurrentLog shutdown of ConcurrentLog.Worker: terminated

Tags

No tags attached.

Attached Files

Relationships

Notes
(0001247) tybor (reporter) 2016-06-08 23:11	Somehow solved. Changing "javastart_Xmx=Xmx600m" into "javastart_Xmx=Xmx2000m" and "javastart_Xms=Xms90m" into "javastart_Xms=Xms200m" made it start.

(0001505) ktplulo (reporter) 2020-03-02 19:12 edited on: 2020-03-02 20:08	I have this issue with 1.922/9964 now. The workaround above is very short-term, because YaCy breaks again soon. Hundreds of megabytes are enough for a few days during crawling. INDEX: 25 G INDEX/SEGMENTS: 24 G HTCACHE: 7.2 G HTCACHE/file.array: 6.8 G Documents: 1.3 M DHT words: 6.2 M javastart_Xms=Xms130M I had to change javastart_Xmm from Xmx1300m to Xmx1400m.

Issue History
Date Modified	Username	Field	Change
2016-06-08 23:04	tybor	New Issue
2016-06-08 23:11	tybor	Note Added: 0001247
2020-03-02 19:12	ktplulo	Note Added: 0001505
2020-03-02 19:17	ktplulo	Note Edited: 0001505	View Revisions
2020-03-02 19:18	ktplulo	Note Edited: 0001505	View Revisions
2020-03-02 19:18	ktplulo	Note Edited: 0001505	View Revisions
2020-03-02 19:30	ktplulo	Note Edited: 0001505	View Revisions
2020-03-02 20:08	ktplulo	Note Edited: 0001505	View Revisions