YaCy-Bugtracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000331YaCy[All Projects] Generalpublic2014-01-08 03:392015-09-01 23:58
Reportersmokingwheels 
Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
StatusnewResolutionopen 
ETAnone 
PlatformX86 OSWindowsOS Version2003
Product VersionYaCy 1.6 
Target VersionFixed in Version 
Summary0000331: Front End Hangs when Searching too oftern
DescriptionWhen Java VM Tops out Front end is not responsive. Errors in log file att
I do a search every 4.3 seconds and in the performance window the blue line flattens out and the search interface hangs.
The install of YaCy was very small eg not many doc's.

I have another install with 5.8 million Docs and similar thing happens with a search interval of 10.3 seconds about 4.6 Gig on Hdd

I have put some of the stuff on FTP Server look in P4 SCSI then yacy3 for copy of complete install the Y folder is the 4.6 G YaCy install.

FTP://yacy.doomdns.org [^]
Name: anonymous
Pass: anything

Part of Att log file

E 2014/01/08 09:34:44 org.apache.solr.core.SolrCore java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit
    at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2749)
    at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2921)
    at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2896)
    at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:559)
    at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1242)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1221)
    at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
    at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:150)
    at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
    at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:179)
    at net.yacy.cora.federate.solr.connector.SolrServerConnector.commit(SolrServerConnector.java:72)
    at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.close(EmbeddedSolrConnector.java:129)
    at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.close(MirrorSolrConnector.java:120)
    at net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector.close(ConcurrentUpdateSolrConnector.java:294)
    at net.yacy.cora.federate.solr.instance.InstanceMirror.disconnect0(InstanceMirror.java:70)
    at net.yacy.cora.federate.solr.instance.InstanceMirror.close(InstanceMirror.java:108)
    at net.yacy.search.index.Fulltext.close(Fulltext.java:297)
    at net.yacy.search.index.Segment.close(Segment.java:556)
    at net.yacy.search.Switchboard.close(Switchboard.java:1738)
    at net.yacy.yacy.startup(yacy.java:437)
    at net.yacy.yacy.main(yacy.java:671)

E 2014/01/08 09:34:44 org.apache.solr.core.SolrCore java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit
    at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2749)
    at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2921)
    at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2896)
    at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:559)
    at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1242)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1221)
    at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
    at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:150)
    at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
    at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:179)
    at net.yacy.cora.federate.solr.connector.SolrServerConnector.close(SolrServerConnector.java:120)
    at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.close(EmbeddedSolrConnector.java:130)
    at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.close(MirrorSolrConnector.java:120)
    at net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector.close(ConcurrentUpdateSolrConnector.java:294)
    at net.yacy.cora.federate.solr.instance.InstanceMirror.disconnect0(InstanceMirror.java:70)
    at net.yacy.cora.federate.solr.instance.InstanceMirror.close(InstanceMirror.java:108)
    at net.yacy.search.index.Fulltext.close(Fulltext.java:297)
    at net.yacy.search.index.Segment.close(Segment.java:556)
    at net.yacy.search.Switchboard.close(Switchboard.java:1738)
    at net.yacy.yacy.startup(yacy.java:437)
    at net.yacy.yacy.main(yacy.java:671)

W 2014/01/08 09:34:44 StackTrace org.apache.solr.client.solrj.SolrServerException: java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit
org.apache.solr.client.solrj.SolrServerException: org.apache.solr.client.solrj.SolrServerException: java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit
    at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:223)
    at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
    at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:179)
    at net.yacy.cora.federate.solr.connector.SolrServerConnector.close(SolrServerConnector.java:120)
    at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.close(EmbeddedSolrConnector.java:130)
    at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.close(MirrorSolrConnector.java:120)
    at net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector.close(ConcurrentUpdateSolrConnector.java:294)
    at net.yacy.cora.federate.solr.instance.InstanceMirror.disconnect0(InstanceMirror.java:70)
    at net.yacy.cora.federate.solr.instance.InstanceMirror.close(InstanceMirror.java:108)
    at net.yacy.search.index.Fulltext.close(Fulltext.java:297)
    at net.yacy.search.index.Segment.close(Segment.java:556)
    at net.yacy.search.Switchboard.close(Switchboard.java:1738)
    at net.yacy.yacy.startup(yacy.java:437)
    at net.yacy.yacy.main(yacy.java:671)
Caused by: org.apache.solr.client.solrj.SolrServerException: java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit
    at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:155)
    ... 13 more
Caused by: java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit
    at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2749)
    at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2921)
    at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2896)
    at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:559)
    at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1242)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1221)
    at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
    at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:150)
    ... 13 more
Steps To ReproduceSearch one word random first names at 10.3 intervals 5 million docs.
The smaller database a 4.3 second search interval has same effect.
Additional InformationThe Queries log from bigger database. Not sure if the funny charters is meant to be there.
Also when the front end seems to hang it repeats the same search name witch has all ready passed long ago from the keyboard emulator
In FTP Directory of YaCy Logs/Jan 7 PM/LOG

FTP://yacy.doomdns.org [^]
Name: anonymous
Pass: anything
 

20140107210420 115 httpstatus_i:200 AND (images_urlstub_sxt:[* TO *] OR url_file_ext_s:(jpg OR png OR gif) ORcontent_type:(image/*)) AND ((images_text_t:"manga"^10.0) OR (text_t:"manga"))
20140107205851 0 Alejandro
20140107205902 0 Jesus
20140107205912 0 Gerald
20140107205924 0 Ernest
20140107205934 0 Harold
20140107205943 0 Gerard
20140107205955 0 Sonny
20140107210006 0 Stephen
20140107210016 0 Lonnie
20140107210027 0 Gregory
20140107210036 0 Elvin
20140107210047 0 Kirk
20140107210102 0 Hilton
20140107210110 0 Dalton
20140107210122 0 Rufus
20140107210128 0 Dalton
20140107210159 0 Dalton
20140107210134 0 Dalton
20140107210330 0 Dalton
20140107211326 3 nutten köln
20140107210154 0 Dalton
20140107210151 0 Dalton
20140107210154 0 Dalton
20140107210154 0 Dalton
20140107210523 0 Dalton
20140107210525 0 Dalton
20140107210549 0 Dalton
20140107210336 0 Dalton
20140107210148 0 Dalton
20140107210549 0 Dalton
20140107210518 0 Dalton
20140107210314 0 Dalton
20140107210518 0 Dalton
20140107210305 0 Dalton
20140107210659 0 Dalton
20140107211048 0 Dalton
20140107211048 0 Dalton
20140107211048 0 Dalton
20140107211050 0 Stephen
20140107211050 0 Stephen
20140107211050 0 Stephen
20140107211110 0 Ernest
20140107210518 0 Dalton
20140107210523 0 Dalton
20140107211046 0 Dalton
20140107210819 0 Dalton
20140107210454 0 Dalton
20140107210503 0 Dalton
20140107210503 0 Dalton
20140107210918 0 Dalton
20140107211050 0 Sonny
20140107211136 0 Gerard
20140107211050 0 Sonny
20140107211050 0 Sonny
20140107211200 0 Gerard
20140107211136 0 Gerard
20140107211138 0 Gerard
20140107211136 0 Gerard
20140107211136 0 Gerard
20140107211136 0 Gerard
20140107211138 0 Gerard
20140107211139 0 Gerard
20140107211136 0 Gerard
20140107211200 0 Gerard
20140107211559 56705 httpstatus_i:200 AND -url_file_ext_s:(jpg OR png OR gif) AND ((url_paths_sxt:"science"^3.0) OR (synonyms_sxt:"science"^0.5) OR (title:"science"^5.0) OR (text_t:"science"^1.0) OR (host_s:"science"^6.0) OR (h1_txt:"science"^5.0) OR (url_file_name_tokens_t:"science"^4.0) OR (h2_txt:"science"^3.0))
20140107211204 0 Gerard
20140107211050 0 Stephen
20140107210812 0 Gerard
20140107211638 106 httpstatus_i:200 AND (images_urlstub_sxt:[* TO *] OR url_file_ext_s:(jpg OR png OR gif) ORcontent_type:(image/*)) AND ((images_text_t:"jupiter"^10.0) OR (text_t:"jupiter"))
20140107212337 0 httpstatus_i:200 AND -url_file_ext_s:(jpg OR png OR gif) AND ((url_paths_sxt:"الدهشان"^3.0) OR (synonyms_sxt:"الدهشان"^0.5) OR (title:"الدهشان"^5.0) OR (text_t:"الدهشان"^1.0) OR (host_s:"الدهشان"^6.0) OR (h1_txt:"الدهشان"^5.0) OR (url_file_name_tokens_t:"الدهشان"^4.0) OR (h2_txt:"الدهشان"^3.0))
20140107210855 0 Dalton
20140107211050 0 Sonny
20140107211232 0 Andre
20140107211041 0 Dalton
20140107211050 0 Sonny
20140107211136 0 Gerard
20140107210934 0 Dalton
20140107211131 0 Dalton
20140107210812 0 Dalton
20140107210934 0 Dalton
TagsNo tags attached.
Attached Fileslog file icon yacy00.log [^] (484,597 bytes) 2014-01-08 03:39
log file icon yacy01.log [^] (1,049,978 bytes) 2014-01-08 15:26

- Relationships

-  Notes
(0000654)
smokingwheels (reporter)
2014-01-08 15:34
edited on: 2014-01-08 15:36

Continuous searching interval beyond PC capability maybe the cause.
Had error pop up in browser 2 lines "501 Exception occurred:Java Heap Space".
Have been Trying 5 search's then wait was good at 20 second interval on P4 3G.
The Dalton in the search quire log is an error they where all different names.

(0001099)
BuBu (developer)
2015-09-01 23:58

Your looked realy into it (the not responding UI after some load event.... maybe searching or crawling).
For me it improved with a commit from June

https://github.com/yacy/yacy_search_server/commit/e427efbe54bd0b45bb33c74d7e8b8c65f8fc1f6b [^]

Do you experience an improvement here, too ?

- Issue History
Date Modified Username Field Change
2014-01-08 03:39 smokingwheels New Issue
2014-01-08 03:39 smokingwheels File Added: yacy00.log
2014-01-08 15:26 smokingwheels File Added: yacy01.log
2014-01-08 15:34 smokingwheels Note Added: 0000654
2014-01-08 15:36 smokingwheels Note Edited: 0000654 View Revisions
2015-09-01 23:58 BuBu Note Added: 0001099


Copyright © 2000 - 2019 MantisBT Team
Powered by Mantis Bugtracker