Anonymous | Login | Signup for a new account | 2021-01-27 23:17 CET | ![]() |
Main | My View | View Issues | Change Log | Roadmap |
View Issue Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||||||||
ID | Project | Category | View Status | Date Submitted | Last Update | ||||||||
0000771 | YaCy | [All Projects] General | public | 2017-10-09 07:43 | 2019-07-28 08:38 | ||||||||
Reporter | smokingwheels | ||||||||||||
Assigned To | |||||||||||||
Priority | normal | Severity | minor | Reproducibility | have not tried | ||||||||
Status | new | Resolution | open | ||||||||||
ETA | none | ||||||||||||
Platform | OS | OS Version | |||||||||||
Product Version | YaCy 1.9 | ||||||||||||
Target Version | Fixed in Version | ||||||||||||
Summary | 0000771: Blocked threads when crawling. | ||||||||||||
Description | I was profiling my Server with YourKit-JavaProfiler eval software. I was crawling some random sites with my modified code and then reverted back to Github version 1.921/9410. Both had blocked threads but the Github version had more and my server did a fatal exception dump for reason unknown to me. I will attach it to a comment. | ||||||||||||
Additional Information | Possible Potential deadlock: frozen threads found in both versions when monitoring. | ||||||||||||
Tags | No tags attached. | ||||||||||||
Attached Files | ![]() Greetings, Thank you for trying YourKit Java Profiler. Your evaluation license key: With My extra comments on *.java and *.js. Files *.css slightly optimized Potential deadlock: frozen threads found It seems that the following threads have not changed their stack for more than 10 seconds. These threads are possibly (but not necessarily!) in a deadlock or hung. storeDocumentIndex_pool-1-thread-43 <--- Frozen for at least 14s net.yacy.cora.federate.solr.connector.SolrServerConnector.add(SolrInputDocument) SolrServerConnector.java:237 net.yacy.cora.federate.solr.connector.MirrorSolrConnector.add(SolrInputDocument) MirrorSolrConnector.java:228 net.yacy.search.index.Fulltext.putDocument(SolrInputDocument) Fulltext.java:330 net.yacy.search.index.Segment.putDocument(SolrInputDocument) Segment.java:510 net.yacy.search.index.Segment.storeDocument(DigestURL, DigestURL, Map, CrawlProfile, ResponseHeader, Document, Condenser, SearchEvent, String, boolean, String, String) Segment.java:553 net.yacy.search.Switchboard.storeDocumentIndex(Response, Map, Document, Condenser, SearchEvent, String) Switchboard.java:2739 net.yacy.search.Switchboard.storeDocumentIndex(IndexingQueueEntry) Switchboard.java:2689 sun.reflect.GeneratedMethodAccessor8.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:109 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:124 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Potential deadlock: frozen threads found It seems that the following threads have not changed their stack for more than 10 seconds. These threads are possibly (but not necessarily!) in a deadlock or hung. commitScheduler-15-thread-1 <--- Frozen for at least 17s sun.nio.ch.FileDispatcherImpl.force0(FileDescriptor, boolean) FileDispatcherImpl.java (native) sun.nio.ch.FileDispatcherImpl.force(FileDescriptor, boolean) FileDispatcherImpl.java:76 sun.nio.ch.FileChannelImpl.force(boolean) FileChannelImpl.java:388 org.apache.lucene.util.IOUtils.fsync(Path, boolean) IOUtils.java:472 org.apache.lucene.store.FSDirectory.fsync(String) FSDirectory.java:327 org.apache.lucene.store.FSDirectory.sync(Collection) FSDirectory.java:285 org.apache.lucene.store.LockValidatingDirectoryWrapper.sync(Collection) LockValidatingDirectoryWrapper.java:68 org.apache.lucene.index.IndexWriter.startCommit(SegmentInfos) IndexWriter.java:4724 org.apache.lucene.index.IndexWriter.prepareCommitInternal(boolean[]) IndexWriter.java:3085 org.apache.lucene.index.IndexWriter.commitInternal(MergePolicy) IndexWriter.java:3244 org.apache.lucene.index.IndexWriter.commit() IndexWriter.java:3207 org.apache.solr.update.DirectUpdateHandler2.commit(CommitUpdateCommand) DirectUpdateHandler2.java:644 org.apache.solr.update.CommitTracker.run() CommitTracker.java:217 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor$ScheduledFutureTask) ScheduledThreadPoolExecutor.java:180 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() ScheduledThreadPoolExecutor.java:293 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 10:42:53 AM. Uptime is 15m 33s 800ms. storeDocumentIndex_pool-1-thread-158 State: BLOCKED CPU usage on sample: 0ms net.yacy.cora.federate.solr.connector.SolrServerConnector.add(SolrInputDocument) SolrServerConnector.java:237 net.yacy.cora.federate.solr.connector.MirrorSolrConnector.add(SolrInputDocument) MirrorSolrConnector.java:228 net.yacy.search.index.Fulltext.putDocument(SolrInputDocument) Fulltext.java:330 net.yacy.search.index.Segment.putDocument(SolrInputDocument) Segment.java:510 net.yacy.search.index.Segment.storeDocument(DigestURL, DigestURL, Map, CrawlProfile, ResponseHeader, Document, Condenser, SearchEvent, String, boolean, String, String) Segment.java:553 net.yacy.search.Switchboard.storeDocumentIndex(Response, Map, Document, Condenser, SearchEvent, String) Switchboard.java:2739 net.yacy.search.Switchboard.storeDocumentIndex(IndexingQueueEntry) Switchboard.java:2689 sun.reflect.GeneratedMethodAccessor13.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:109 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:124 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 10:54:53 AM. Uptime is 27m 34s 800ms. storeDocumentIndex_pool-1-thread-158 State: BLOCKED CPU usage on sample: 72ms java.util.Collections$SynchronizedMap.get(Object) Collections.java:2584 net.yacy.cora.storage.SimpleARC.get(Object) SimpleARC.java:209 net.yacy.cora.storage.ConcurrentARC.get(Object) ConcurrentARC.java:173 net.yacy.cora.order.Digest.encodeMD5Raw(String) Digest.java:135 net.yacy.cora.document.id.DigestURL.urlHashComputation() DigestURL.java:287 net.yacy.cora.document.id.DigestURL.hash() DigestURL.java:230 net.yacy.search.index.Segment.storeDocument(DigestURL, DigestURL, Map, CrawlProfile, ResponseHeader, Document, Condenser, SearchEvent, String, boolean, String, String) Segment.java:582 net.yacy.search.Switchboard.storeDocumentIndex(Response, Map, Document, Condenser, SearchEvent, String) Switchboard.java:2739 net.yacy.search.Switchboard.storeDocumentIndex(IndexingQueueEntry) Switchboard.java:2689 sun.reflect.GeneratedMethodAccessor13.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:109 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:124 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 10:55:07 AM. Uptime is 27m 48s 100ms. IODispatcher State: WAITING CPU usage on sample: 0ms sun.misc.Unsafe.park(boolean, long) Unsafe.java (native) java.util.concurrent.locks.LockSupport.park(Object) LockSupport.java:175 java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() AbstractQueuedSynchronizer.java:836 java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(int) AbstractQueuedSynchronizer.java:997 java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(int) AbstractQueuedSynchronizer.java:1304 java.util.concurrent.Semaphore.acquire() Semaphore.java:312 net.yacy.kelondro.rwi.IODispatcher.run() IODispatcher.java:198 Stacks at 2017-10-09 10:58:36 AM. Uptime is 31m 17s 100ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 0ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:169 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:138 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:128 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:185 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:232 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:471 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:300 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:254 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:330 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:984 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:100 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:172 Stacks at 2017-10-09 10:59:12 AM. Uptime is 31m 53s 300ms. Embedded solr query: q={!cache=false raw f=id}eTk9zJ80aHL6&rows=1&start=0&fl=id,load_date_dt&defType=edismax&qf=text_t^1.0 State: WAITING CPU usage on sample: 33ms sun.misc.Unsafe.park(boolean, long) Unsafe.java (native) java.util.concurrent.locks.LockSupport.park(Object) LockSupport.java:175 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() AbstractQueuedSynchronizer.java:2039 java.util.concurrent.LinkedBlockingQueue.take() LinkedBlockingQueue.java:442 net.yacy.kelondro.workflow.WorkflowProcessor.take() WorkflowProcessor.java:123 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:119 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:00:17 AM. Uptime is 32m 58s 100ms. Embedded solr query: q={!cache=false raw f=id}eTk9zJ80aHL6&rows=1&start=0&fl=id,load_date_dt&defType=edismax&qf=text_t^1.0 State: BLOCKED CPU usage on sample: 11ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:291 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:163 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:356 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:134 sun.reflect.GeneratedMethodAccessor3.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:109 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:124 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:03:52 AM. Uptime is 36m 33s 400ms. Embedded solr query: q={!cache=false raw f=id}YLsTkp6VLpe6&rows=1&start=0&fl=id,load_date_dt&defType=edismax&qf=text_t^1.0 State: BLOCKED CPU usage on sample: 12ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:291 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:163 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:356 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:134 sun.reflect.GeneratedMethodAccessor3.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:109 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:124 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Restored code to github 1.921/9410 11:14 am Stacks at 2017-10-09 11:17:37 AM. Uptime is 6m 34s 200ms. job_pool-1-thread-88 State: BLOCKED CPU usage on sample: 87ms net.yacy.kelondro.index.BufferedObjectIndex.has(byte[]) BufferedObjectIndex.java:178 net.yacy.crawler.HostQueue.has(byte[]) HostQueue.java:369 net.yacy.crawler.HostBalancer.has(byte[]) HostBalancer.java:205 net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:245 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:184 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:390 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:135 sun.reflect.GeneratedMethodAccessor4.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:18:07 AM. Uptime is 7m 4s 600ms. job_pool-1-thread-87 State: BLOCKED CPU usage on sample: 27ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:248 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:184 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:390 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:135 sun.reflect.GeneratedMethodAccessor4.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:18:44 AM. Uptime is 7m 41s 400ms. job_pool-1-thread-87 State: BLOCKED CPU usage on sample: 32ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:248 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:184 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:390 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:135 sun.reflect.GeneratedMethodAccessor4.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:18:46 AM. Uptime is 7m 43s 400ms. qtp1281414889-113 State: BLOCKED CPU usage on sample: 4ms net.yacy.kelondro.blob.ArrayStack.sizes() ArrayStack.java:508 net.yacy.kelondro.rwi.ReferenceContainerArray.sizes() ReferenceContainerArray.java:87 net.yacy.kelondro.rwi.IndexCell.sizes() IndexCell.java:622 net.yacy.kelondro.rwi.IndexCell.sizesMax() IndexCell.java:631 net.yacy.search.index.Segment.RWICount() Segment.java:343 net.yacy.search.Switchboard.updateMySeed() Switchboard.java:4172 status_p.respond(RequestHeader, serverObjects, serverSwitch) status_p.java:63 sun.reflect.GeneratedMethodAccessor10.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.http.servlets.YaCyDefaultServlet.invokeServlet(File, RequestHeader, serverObjects) YaCyDefaultServlet.java:673 net.yacy.http.servlets.YaCyDefaultServlet.handleTemplate(String, HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:884 net.yacy.http.servlets.YaCyDefaultServlet.doGet(HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:315 javax.servlet.http.HttpServlet.service(HttpServletRequest, HttpServletResponse) HttpServlet.java:687 javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse) HttpServlet.java:790 org.eclipse.jetty.servlet.ServletHolder.handle(Request, ServletRequest, ServletResponse) ServletHolder.java:841 org.eclipse.jetty.servlet.ServletHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:535 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:143 org.eclipse.jetty.security.SecurityHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) SecurityHandler.java:513 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:190 org.eclipse.jetty.server.session.SessionHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1595 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:188 org.eclipse.jetty.server.handler.ContextHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1253 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:168 org.eclipse.jetty.servlet.ServletHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:473 org.eclipse.jetty.server.session.SessionHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1564 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:166 org.eclipse.jetty.server.handler.ContextHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1155 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:141 org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandlerCollection.java:219 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 net.yacy.http.CrashProtectionHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) CrashProtectionHandler.java:33 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.Server.handle(HttpChannel) Server.java:564 org.eclipse.jetty.server.HttpChannel.handle() HttpChannel.java:317 org.eclipse.jetty.server.HttpConnection.onFillable() HttpConnection.java:251 org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded() AbstractConnection.java:279 org.eclipse.jetty.io.FillInterest.fillable() FillInterest.java:110 org.eclipse.jetty.io.ChannelEndPoint$2.run() ChannelEndPoint.java:124 org.eclipse.jetty.util.thread.Invocable.invokePreferred(Runnable, Invocable$InvocationType) Invocable.java:128 org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Runnable) Invocable.java:222 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce() EatWhatYouKill.java:294 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run() EatWhatYouKill.java:199 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Runnable) QueuedThreadPool.java:673 org.eclipse.jetty.util.thread.QueuedThreadPool$2.run() QueuedThreadPool.java:591 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:19:46 AM. Uptime is 8m 43s 300ms. qtp1281414889-205 State: BLOCKED CPU usage on sample: 3ms net.yacy.kelondro.index.BufferedObjectIndex.size() BufferedObjectIndex.java:151 net.yacy.crawler.HostQueue.size() HostQueue.java:381 net.yacy.crawler.HostBalancer.size() HostBalancer.java:212 net.yacy.crawler.data.NoticedURL.size() NoticedURL.java:130 net.yacy.search.Switchboard.updateMySeed() Switchboard.java:4170 status_p.respond(RequestHeader, serverObjects, serverSwitch) status_p.java:63 sun.reflect.GeneratedMethodAccessor10.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.http.servlets.YaCyDefaultServlet.invokeServlet(File, RequestHeader, serverObjects) YaCyDefaultServlet.java:673 net.yacy.http.servlets.YaCyDefaultServlet.handleTemplate(String, HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:884 net.yacy.http.servlets.YaCyDefaultServlet.doGet(HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:315 javax.servlet.http.HttpServlet.service(HttpServletRequest, HttpServletResponse) HttpServlet.java:687 javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse) HttpServlet.java:790 org.eclipse.jetty.servlet.ServletHolder.handle(Request, ServletRequest, ServletResponse) ServletHolder.java:841 org.eclipse.jetty.servlet.ServletHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:535 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:143 org.eclipse.jetty.security.SecurityHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) SecurityHandler.java:513 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:190 org.eclipse.jetty.server.session.SessionHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1595 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:188 org.eclipse.jetty.server.handler.ContextHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1253 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:168 org.eclipse.jetty.servlet.ServletHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:473 org.eclipse.jetty.server.session.SessionHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1564 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:166 org.eclipse.jetty.server.handler.ContextHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1155 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:141 org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandlerCollection.java:219 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 net.yacy.http.CrashProtectionHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) CrashProtectionHandler.java:33 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.Server.handle(HttpChannel) Server.java:564 org.eclipse.jetty.server.HttpChannel.handle() HttpChannel.java:317 org.eclipse.jetty.server.HttpConnection.onFillable() HttpConnection.java:251 org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded() AbstractConnection.java:279 org.eclipse.jetty.io.FillInterest.fillable() FillInterest.java:110 org.eclipse.jetty.io.ChannelEndPoint$2.run() ChannelEndPoint.java:124 org.eclipse.jetty.util.thread.Invocable.invokePreferred(Runnable, Invocable$InvocationType) Invocable.java:128 org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Runnable) Invocable.java:222 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce() EatWhatYouKill.java:294 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run() EatWhatYouKill.java:199 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Runnable) QueuedThreadPool.java:673 org.eclipse.jetty.util.thread.QueuedThreadPool$2.run() QueuedThreadPool.java:591 java.lang.Thread.run() Thread.java:748 Potential deadlock: frozen threads found It seems that the following threads have not changed their stack for more than 10 seconds. These threads are possibly (but not necessarily!) in a deadlock or hung. InetAddress.getByName_pool-1-thread-462 <--- Frozen for at least 13s java.net.Inet6AddressImpl.lookupAllHostAddr(String) Inet6AddressImpl.java (native) java.net.InetAddress$2.lookupAllHostAddr(String) InetAddress.java:928 java.net.InetAddress.getAddressesFromNameService(String, InetAddress) InetAddress.java:1323 java.net.InetAddress.getAllByName0(String, InetAddress, boolean) InetAddress.java:1276 java.net.InetAddress.getAllByName(String, InetAddress) InetAddress.java:1192 java.net.InetAddress.getAllByName(String) InetAddress.java:1126 java.net.InetAddress.getByName(String) InetAddress.java:1076 net.yacy.cora.protocol.Domains$2.call() Domains.java:960 net.yacy.cora.protocol.Domains$2.call() Domains.java:957 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:21:23 AM. Uptime is 10m 20s 500ms. qtp1281414889-217 State: BLOCKED CPU usage on sample: 0ms java.io.ExpiringCache.get(String) ExpiringCache.java:75 java.io.UnixFileSystem.canonicalize(String) UnixFileSystem.java:152 java.io.File.getCanonicalPath() File.java:618 net.yacy.http.servlets.YaCyDefaultServlet.rewriteClassFile(File) YaCyDefaultServlet.java:763 net.yacy.http.servlets.YaCyDefaultServlet.handleTemplate(String, HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:841 net.yacy.http.servlets.YaCyDefaultServlet.doGet(HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:315 javax.servlet.http.HttpServlet.service(HttpServletRequest, HttpServletResponse) HttpServlet.java:687 javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse) HttpServlet.java:790 org.eclipse.jetty.servlet.ServletHolder.handle(Request, ServletRequest, ServletResponse) ServletHolder.java:841 org.eclipse.jetty.servlet.ServletHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:535 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:143 org.eclipse.jetty.security.SecurityHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) SecurityHandler.java:524 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:190 org.eclipse.jetty.server.session.SessionHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1595 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:188 org.eclipse.jetty.server.handler.ContextHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1253 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:168 org.eclipse.jetty.servlet.ServletHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:473 org.eclipse.jetty.server.session.SessionHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1564 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:166 org.eclipse.jetty.server.handler.ContextHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1155 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:141 org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandlerCollection.java:219 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 net.yacy.http.CrashProtectionHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) CrashProtectionHandler.java:33 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.Server.handle(HttpChannel) Server.java:564 org.eclipse.jetty.server.HttpChannel.handle() HttpChannel.java:317 org.eclipse.jetty.server.HttpConnection.onFillable() HttpConnection.java:251 org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded() AbstractConnection.java:279 org.eclipse.jetty.io.FillInterest.fillable() FillInterest.java:110 org.eclipse.jetty.io.ChannelEndPoint$2.run() ChannelEndPoint.java:124 org.eclipse.jetty.util.thread.Invocable.invokePreferred(Runnable, Invocable$InvocationType) Invocable.java:128 org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Runnable) Invocable.java:222 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce() EatWhatYouKill.java:294 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run() EatWhatYouKill.java:199 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Runnable) QueuedThreadPool.java:673 org.eclipse.jetty.util.thread.QueuedThreadPool$2.run() QueuedThreadPool.java:591 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:22:02 AM. Uptime is 10m 59s 200ms. BusyThread Switchboard.dhtTransferJob [DAEMON] State: BLOCKED CPU usage on sample: 177ms net.yacy.kelondro.blob.ArrayStack.keys(boolean, byte[]) ArrayStack.java:542 net.yacy.kelondro.rwi.ReferenceContainerArray$ReferenceContainerIterator.<init>(ReferenceContainerArray, byte[], boolean, boolean) ReferenceContainerArray.java:134 net.yacy.kelondro.rwi.ReferenceContainerArray.referenceContainerIterator(byte[], boolean, boolean) ReferenceContainerArray.java:114 net.yacy.kelondro.rwi.IndexCell.referenceContainerIterator(byte[], boolean, boolean, boolean) IndexCell.java:562 net.yacy.peers.Dispatcher.selectContainers(byte[], byte[], int, int, int, boolean) Dispatcher.java:175 net.yacy.peers.Dispatcher.selectContainers(byte[], byte[], int, int, int) Dispatcher.java:158 net.yacy.peers.Dispatcher.selectContainersEnqueueToBuffer(byte[], byte[], int, int, int) Dispatcher.java:306 net.yacy.search.Switchboard.dhtTransferJob() Switchboard.java:3935 net.yacy.search.Switchboard$12.jobImpl() Switchboard.java:1192 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:64 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:215 Stacks at 2017-10-09 11:22:34 AM. Uptime is 11m 30s 800ms. qtp1281414889-121 State: BLOCKED CPU usage on sample: 4ms org.apache.lucene.index.IndexWriter.isClosed() IndexWriter.java:4891 org.apache.lucene.index.StandardDirectoryReader.isCurrent() StandardDirectoryReader.java:350 org.apache.lucene.index.FilterDirectoryReader.isCurrent() FilterDirectoryReader.java:124 <2 recursive calls> org.apache.solr.handler.admin.LukeRequestHandler$$Lambda$182.get() org.apache.solr.handler.admin.LukeRequestHandler.closeSafe(LukeRequestHandler$IOSupplier) LukeRequestHandler.java:605 org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(DirectoryReader) LukeRequestHandler.java:582 org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(SolrQueryRequest, SolrQueryResponse) LukeRequestHandler.java:138 org.apache.solr.handler.RequestHandlerBase.handleRequest(SolrQueryRequest, SolrQueryResponse) RequestHandlerBase.java:173 org.apache.solr.core.SolrCore.execute(SolrRequestHandler, SolrQueryRequest, SolrQueryResponse) SolrCore.java:2477 org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(SolrRequest, String) EmbeddedSolrServer.java:179 org.apache.solr.client.solrj.SolrRequest.process(SolrClient, String) SolrRequest.java:160 org.apache.solr.client.solrj.SolrRequest.process(SolrClient) SolrRequest.java:177 net.yacy.cora.federate.solr.connector.SolrServerConnector.getIndexBrowser(boolean) SolrServerConnector.java:421 net.yacy.cora.federate.solr.connector.SolrServerConnector.getSegmentCount() SolrServerConnector.java:345 net.yacy.cora.federate.solr.connector.MirrorSolrConnector.getSegmentCount() MirrorSolrConnector.java:133 status_p.respond(RequestHeader, serverObjects, serverSwitch) status_p.java:87 sun.reflect.GeneratedMethodAccessor10.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.http.servlets.YaCyDefaultServlet.invokeServlet(File, RequestHeader, serverObjects) YaCyDefaultServlet.java:673 net.yacy.http.servlets.YaCyDefaultServlet.handleTemplate(String, HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:884 net.yacy.http.servlets.YaCyDefaultServlet.doGet(HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:315 javax.servlet.http.HttpServlet.service(HttpServletRequest, HttpServletResponse) HttpServlet.java:687 javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse) HttpServlet.java:790 org.eclipse.jetty.servlet.ServletHolder.handle(Request, ServletRequest, ServletResponse) ServletHolder.java:841 org.eclipse.jetty.servlet.ServletHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:535 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:143 org.eclipse.jetty.security.SecurityHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) SecurityHandler.java:513 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:190 org.eclipse.jetty.server.session.SessionHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1595 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:188 org.eclipse.jetty.server.handler.ContextHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1253 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:168 org.eclipse.jetty.servlet.ServletHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:473 org.eclipse.jetty.server.session.SessionHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1564 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:166 org.eclipse.jetty.server.handler.ContextHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1155 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:141 org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandlerCollection.java:219 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 net.yacy.http.CrashProtectionHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) CrashProtectionHandler.java:33 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.Server.handle(HttpChannel) Server.java:564 org.eclipse.jetty.server.HttpChannel.handle() HttpChannel.java:317 org.eclipse.jetty.server.HttpConnection.onFillable() HttpConnection.java:251 org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded() AbstractConnection.java:279 org.eclipse.jetty.io.FillInterest.fillable() FillInterest.java:110 org.eclipse.jetty.io.ChannelEndPoint$2.run() ChannelEndPoint.java:124 org.eclipse.jetty.util.thread.Invocable.invokePreferred(Runnable, Invocable$InvocationType) Invocable.java:128 org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Runnable) Invocable.java:222 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce() EatWhatYouKill.java:294 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run() EatWhatYouKill.java:199 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Runnable) QueuedThreadPool.java:673 org.eclipse.jetty.util.thread.QueuedThreadPool$2.run() QueuedThreadPool.java:591 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:23:15 AM. Uptime is 12m 12s 200ms. qtp1281414889-112 State: BLOCKED CPU usage on sample: 3ms java.lang.Object.wait(long) Object.java (native) java.lang.Object.wait() Object.java:502 org.eclipse.jetty.server.HttpInput.blockForContent() HttpInput.java:541 org.eclipse.jetty.server.HttpInputOverHTTP.blockForContent() HttpInputOverHTTP.java:42 org.eclipse.jetty.server.HttpInput$1.blockForContent(HttpInput) HttpInput.java:1062 org.eclipse.jetty.server.HttpInput.read(byte[], int, int) HttpInput.java:304 org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable() MultipartStream.java:999 org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(byte[], int, int) MultipartStream.java:903 java.io.FilterInputStream.read(byte[], int, int) FilterInputStream.java:133 org.apache.commons.fileupload.util.LimitedInputStream.read(byte[], int, int) LimitedInputStream.java:134 java.io.FilterInputStream.read(byte[]) FilterInputStream.java:107 org.apache.commons.fileupload.util.Streams.copy(InputStream, OutputStream, boolean, byte[]) Streams.java:100 org.apache.commons.fileupload.util.Streams.copy(InputStream, OutputStream, boolean) Streams.java:70 org.apache.commons.fileupload.FileUploadBase.parseRequest(RequestContext) FileUploadBase.java:347 org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(HttpServletRequest) ServletFileUpload.java:115 net.yacy.http.servlets.YaCyDefaultServlet.parseMultipart(HttpServletRequest, serverObjects) YaCyDefaultServlet.java:1261 net.yacy.http.servlets.YaCyDefaultServlet.handleTemplate(String, HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:874 net.yacy.http.servlets.YaCyDefaultServlet.doGet(HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:315 net.yacy.http.servlets.YaCyDefaultServlet.doPost(HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:377 javax.servlet.http.HttpServlet.service(HttpServletRequest, HttpServletResponse) HttpServlet.java:707 javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse) HttpServlet.java:790 org.eclipse.jetty.servlet.ServletHolder.handle(Request, ServletRequest, ServletResponse) ServletHolder.java:841 org.eclipse.jetty.servlet.ServletHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:535 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:143 org.eclipse.jetty.security.SecurityHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) SecurityHandler.java:524 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:190 org.eclipse.jetty.server.session.SessionHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1595 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:188 org.eclipse.jetty.server.handler.ContextHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1253 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:168 org.eclipse.jetty.servlet.ServletHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:473 org.eclipse.jetty.server.session.SessionHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1564 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:166 org.eclipse.jetty.server.handler.ContextHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1155 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:141 org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandlerCollection.java:219 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 net.yacy.http.CrashProtectionHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) CrashProtectionHandler.java:33 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.Server.handle(HttpChannel) Server.java:564 org.eclipse.jetty.server.HttpChannel.handle() HttpChannel.java:317 org.eclipse.jetty.server.HttpConnection.onFillable() HttpConnection.java:251 org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded() AbstractConnection.java:279 org.eclipse.jetty.io.FillInterest.fillable() FillInterest.java:110 org.eclipse.jetty.io.ChannelEndPoint$2.run() ChannelEndPoint.java:124 org.eclipse.jetty.util.thread.Invocable.invokePreferred(Runnable, Invocable$InvocationType) Invocable.java:128 org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Runnable) Invocable.java:222 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce() EatWhatYouKill.java:294 org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run() EatWhatYouKill.java:199 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Runnable) QueuedThreadPool.java:673 org.eclipse.jetty.util.thread.QueuedThreadPool$2.run() QueuedThreadPool.java:591 java.lang.Thread.run() Thread.java:748 Lots of these Stacks at 2017-10-09 11:23:24 AM. Uptime is 12m 21s 200ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 0ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:163 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:125 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:125 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:216 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:285 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:455 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:420 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:332 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:285 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:307 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1141 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:64 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:215 Stacks at 2017-10-09 11:23:30 AM. Uptime is 12m 27s 200ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 0ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:163 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:125 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:125 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:216 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:285 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:455 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:420 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:332 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:285 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:307 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1141 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:64 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:215 Stacks at 2017-10-09 11:24:38 AM. Uptime is 13m 34s 900ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 5ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:163 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:125 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:125 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:216 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:285 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:455 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:420 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:332 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:285 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:307 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1141 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:64 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:215 Stacks at 2017-10-09 11:24:42 AM. Uptime is 13m 39s 200ms. pTHuKZW-DefI_pool-1-thread-21 State: BLOCKED CPU usage on sample: 3ms net.yacy.kelondro.blob.HeapReader.containsKey(byte[]) HeapReader.java:391 net.yacy.kelondro.blob.ArrayStack$1.call() ArrayStack.java:598 net.yacy.kelondro.blob.ArrayStack$1.call() ArrayStack.java:595 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:25:00 AM. Uptime is 13m 57s 400ms. job_pool-1-thread-86 State: BLOCKED CPU usage on sample: 53ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:248 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:184 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:390 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:135 sun.reflect.GeneratedMethodAccessor4.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:25:21 AM. Uptime is 14m 17s 900ms. job_pool-1-thread-86 State: BLOCKED CPU usage on sample: 12ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:248 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:184 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:390 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:135 sun.reflect.GeneratedMethodAccessor4.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:25:00 AM. Uptime is 13m 57s 400ms. job_pool-1-thread-88 State: BLOCKED CPU usage on sample: 23ms net.yacy.kelondro.index.RowHandleSet.has(byte[]) RowHandleSet.java:204 net.yacy.repository.Blacklist.isListed(Blacklist$BlacklistType, DigestURL) Blacklist.java:561 net.yacy.crawler.CrawlStacker.checkAcceptanceChangeable(DigestURL, CrawlProfile, int) CrawlStacker.java:486 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:349 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:135 sun.reflect.GeneratedMethodAccessor4.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:25:21 AM. Uptime is 14m 17s 900ms. job_pool-1-thread-88 State: BLOCKED CPU usage on sample: 8ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:248 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:184 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:390 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:135 sun.reflect.GeneratedMethodAccessor4.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 11:25:00 AM. Uptime is 13m 57s 400ms. Thread-934 [DAEMON] State: BLOCKED CPU usage on sample: 0ms net.yacy.cora.federate.solr.connector.SolrServerConnector.add(SolrInputDocument) SolrServerConnector.java:211 net.yacy.cora.federate.solr.connector.MirrorSolrConnector.add(SolrInputDocument) MirrorSolrConnector.java:204 net.yacy.search.index.ErrorCache.push(DigestURL, int, CrawlProfile, FailCategory, String, int) ErrorCache.java:122 net.yacy.crawler.retrieval.HTTPLoader.load(Request, CrawlProfile, int, int, Blacklist$BlacklistType, ClientIdentification$Agent) HTTPLoader.java:418 net.yacy.crawler.retrieval.HTTPLoader.load(Request, CrawlProfile, int, Blacklist$BlacklistType, ClientIdentification$Agent) HTTPLoader.java:83 net.yacy.repository.LoaderDispatcher.loadInternal(Request, CacheStrategy, int, Blacklist$BlacklistType, ClientIdentification$Agent) LoaderDispatcher.java:242 net.yacy.repository.LoaderDispatcher.load(Request, CacheStrategy, int, Blacklist$BlacklistType, ClientIdentification$Agent) LoaderDispatcher.java:180 net.yacy.repository.LoaderDispatcher.load(Request, CacheStrategy, Blacklist$BlacklistType, ClientIdentification$Agent) LoaderDispatcher.java:151 net.yacy.crawler.data.CrawlQueues$Loader.run() CrawlQueues.java:749 Lots of these and crawler slowed down Stacks at 2017-10-09 11:54:00 AM. Uptime is 42m 57s 0ms. Thread-1159 State: BLOCKED CPU usage on sample: 0ms net.yacy.cora.federate.solr.connector.SolrServerConnector.add(SolrInputDocument) SolrServerConnector.java:211 net.yacy.cora.federate.solr.connector.MirrorSolrConnector.add(SolrInputDocument) MirrorSolrConnector.java:204 net.yacy.search.index.ErrorCache.push(DigestURL, int, CrawlProfile, FailCategory, String, int) ErrorCache.java:122 net.yacy.crawler.data.CrawlQueues$Loader.run() CrawlQueues.java:776 Stacks at 2017-10-09 12:00:33 PM. Uptime is 49m 30s 100ms. qtp1281414889-3308 State: BLOCKED CPU usage on sample: 0ms java.lang.ClassLoader.defineClass1(String, byte[], int, int, ProtectionDomain, String) ClassLoader.java (native) java.lang.ClassLoader.defineClass(String, byte[], int, int, ProtectionDomain) ClassLoader.java:763 java.lang.ClassLoader.defineClass(String, byte[], int, int) ClassLoader.java:642 net.yacy.server.serverClassLoader.loadClass(File) serverClassLoader.java:93 net.yacy.http.servlets.YaCyDefaultServlet.rewriteMethod(File) YaCyDefaultServlet.java:793 net.yacy.http.servlets.YaCyDefaultServlet.invokeServlet(File, RequestHeader, serverObjects) YaCyDefaultServlet.java:673 net.yacy.http.servlets.YaCyDefaultServlet.handleTemplate(String, HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:884 net.yacy.http.servlets.YaCyDefaultServlet.doGet(HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:315 net.yacy.http.servlets.YaCyDefaultServlet.doPost(HttpServletRequest, HttpServletResponse) YaCyDefaultServlet.java:377 javax.servlet.http.HttpServlet.service(HttpServletRequest, HttpServletResponse) HttpServlet.java:707 javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse) HttpServlet.java:790 org.eclipse.jetty.servlet.ServletHolder.handle(Request, ServletRequest, ServletResponse) ServletHolder.java:841 org.eclipse.jetty.servlet.ServletHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:535 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:143 org.eclipse.jetty.security.SecurityHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) SecurityHandler.java:524 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:190 org.eclipse.jetty.server.session.SessionHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1595 org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:188 org.eclipse.jetty.server.handler.ContextHandler.doHandle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1253 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:168 org.eclipse.jetty.servlet.ServletHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ServletHandler.java:473 org.eclipse.jetty.server.session.SessionHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) SessionHandler.java:1564 org.eclipse.jetty.server.handler.ScopedHandler.nextScope(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:166 org.eclipse.jetty.server.handler.ContextHandler.doScope(String, Request, HttpServletRequest, HttpServletResponse) ContextHandler.java:1155 org.eclipse.jetty.server.handler.ScopedHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) ScopedHandler.java:141 org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(String, Request, HttpServletRequest, HttpServletResponse) ContextHandlerCollection.java:219 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 net.yacy.http.CrashProtectionHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) CrashProtectionHandler.java:33 org.eclipse.jetty.server.handler.HandlerWrapper.handle(String, Request, HttpServletRequest, HttpServletResponse) HandlerWrapper.java:132 org.eclipse.jetty.server.Server.handle(HttpChannel) Server.java:564 org.eclipse.jetty.server.HttpChannel.handle() HttpChannel.java:317 org.eclipse.jetty.server.HttpConnection.onFillable() HttpConnection.java:251 org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded() AbstractConnection.java:279 org.eclipse.jetty.io.FillInterest.fillable() FillInterest.java:110 org.eclipse.jetty.io.ChannelEndPoint$2.run() ChannelEndPoint.java:124 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Runnable) QueuedThreadPool.java:673 org.eclipse.jetty.util.thread.QueuedThreadPool$2.run() QueuedThreadPool.java:591 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 12:02:52 PM. Uptime is 51m 49s 200ms. storeDocumentIndex_pool-1-thread-93 State: BLOCKED CPU usage on sample: 29ms java.util.logging.FileHandler.publish(LogRecord) FileHandler.java:679 java.util.logging.Logger.log(LogRecord) Logger.java:738 org.slf4j.impl.JDK14LoggerAdapter.log(String, Level, String, Throwable) JDK14LoggerAdapter.java:582 org.slf4j.impl.JDK14LoggerAdapter.info(String) JDK14LoggerAdapter.java:277 org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.finish() LogUpdateProcessorFactory.java:187 org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(SolrQueryRequest, SolrQueryResponse) ContentStreamHandlerBase.java:78 org.apache.solr.handler.RequestHandlerBase.handleRequest(SolrQueryRequest, SolrQueryResponse) RequestHandlerBase.java:173 org.apache.solr.core.SolrCore.execute(SolrRequestHandler, SolrQueryRequest, SolrQueryResponse) SolrCore.java:2477 org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(SolrRequest, String) EmbeddedSolrServer.java:179 org.apache.solr.client.solrj.SolrRequest.process(SolrClient, String) SolrRequest.java:160 org.apache.solr.client.solrj.SolrClient.add(String, SolrInputDocument, int) SolrClient.java:173 org.apache.solr.client.solrj.SolrClient.add(SolrInputDocument, int) SolrClient.java:190 net.yacy.cora.federate.solr.connector.SolrServerConnector.add(SolrInputDocument) SolrServerConnector.java:213 net.yacy.cora.federate.solr.connector.MirrorSolrConnector.add(SolrInputDocument) MirrorSolrConnector.java:204 net.yacy.search.index.Fulltext.putDocument(SolrInputDocument) Fulltext.java:349 net.yacy.search.index.Segment.putDocument(SolrInputDocument) Segment.java:555 net.yacy.search.index.Segment.storeDocument(DigestURL, DigestURL, Map, CrawlProfile, ResponseHeader, Document, Condenser, SearchEvent, String, boolean, String, String) Segment.java:616 net.yacy.search.Switchboard.storeDocumentIndex(Response, Map, Document, Condenser, SearchEvent, String) Switchboard.java:3252 net.yacy.search.Switchboard.storeDocumentIndex(IndexingQueueEntry) Switchboard.java:3186 sun.reflect.GeneratedMethodAccessor13.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 12:04:32 PM. Uptime is 53m 29s 0ms. condenseDocument_pool-1-thread-80 State: BLOCKED CPU usage on sample: 3ms net.yacy.cora.storage.SimpleARC.get(Object) SimpleARC.java:143 net.yacy.cora.storage.ConcurrentARC.get(Object) ConcurrentARC.java:140 net.yacy.cora.order.Digest.encodeMD5Raw(String) Digest.java:122 net.yacy.cora.document.id.DigestURL.urlHashComputation() DigestURL.java:280 net.yacy.cora.document.id.DigestURL.hash() DigestURL.java:206 net.yacy.cora.document.id.DigestURL.hashCode() DigestURL.java:181 java.util.HashMap.hash(Object) HashMap.java:338 java.util.HashMap.put(Object, Object) HashMap.java:611 net.yacy.document.Document.resortLinks() Document.java:593 net.yacy.document.Document.getImages() Document.java:531 net.yacy.document.Condenser.<init>(Document, VocabularyScraper, boolean, boolean, WordCache, boolean, boolean, int) Condenser.java:83 net.yacy.search.Switchboard.condenseDocument(IndexingQueueEntry) Switchboard.java:3145 sun.reflect.GeneratedMethodAccessor6.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:101 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:82 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Potential deadlock: frozen threads found It seems that the following threads have not changed their stack for more than 10 seconds. These threads are possibly (but not necessarily!) in a deadlock or hung. commitScheduler-15-thread-1 <--- Frozen for at least 1m 1s sun.nio.ch.FileDispatcherImpl.force0(FileDescriptor, boolean) FileDispatcherImpl.java (native) sun.nio.ch.FileDispatcherImpl.force(FileDescriptor, boolean) FileDispatcherImpl.java:76 sun.nio.ch.FileChannelImpl.force(boolean) FileChannelImpl.java:388 org.apache.lucene.util.IOUtils.fsync(Path, boolean) IOUtils.java:472 org.apache.lucene.store.FSDirectory.fsync(String) FSDirectory.java:327 org.apache.lucene.store.FSDirectory.sync(Collection) FSDirectory.java:285 org.apache.lucene.store.LockValidatingDirectoryWrapper.sync(Collection) LockValidatingDirectoryWrapper.java:68 org.apache.lucene.index.IndexWriter.startCommit(SegmentInfos) IndexWriter.java:4724 org.apache.lucene.index.IndexWriter.prepareCommitInternal(boolean[]) IndexWriter.java:3085 org.apache.lucene.index.IndexWriter.commitInternal(MergePolicy) IndexWriter.java:3244 org.apache.lucene.index.IndexWriter.commit() IndexWriter.java:3207 org.apache.solr.update.DirectUpdateHandler2.commit(CommitUpdateCommand) DirectUpdateHandler2.java:644 org.apache.solr.update.CommitTracker.run() CommitTracker.java:217 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor$ScheduledFutureTask) ScheduledThreadPoolExecutor.java:180 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() ScheduledThreadPoolExecutor.java:293 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-09 12:06:19 PM. Uptime is 55m 16s 500ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 52ms net.yacy.kelondro.blob.MapHeap.containsKey(Object) MapHeap.java:242 net.yacy.crawler.data.Cache.has(byte[]) Cache.java:307 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:284 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:455 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:420 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:332 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:282 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:326 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1141 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:64 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:215 Stacks at 2017-10-09 12:06:40 PM. Uptime is 55m 37s 500ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 43ms net.yacy.kelondro.blob.MapHeap.containsKey(Object) MapHeap.java:242 net.yacy.crawler.data.Cache.has(byte[]) Cache.java:307 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:284 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:455 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:420 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:332 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:282 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:326 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1141 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:64 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:215 Stacks at 2017-10-09 12:06:46 PM. Uptime is 55m 42s 700ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 14ms net.yacy.kelondro.blob.MapHeap.containsKey(Object) MapHeap.java:242 net.yacy.crawler.data.Cache.has(byte[]) Cache.java:307 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:284 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:455 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:420 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:332 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:282 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:326 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1141 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:64 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:215 Crawler slow... Stacks at 2017-10-09 12:06:54 PM. Uptime is 55m 51s 400ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 24ms net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:332 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:282 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:326 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1141 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:64 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:215 ![]() ![]() ![]() Just started low speed crawl 25 ppm depth 5 with My Raspberry Pi-Hole blocklist I have 3 days left on yourkit app. I have my source if you need it for line numbers. Stacks at 2017-10-21 02:39:36 PM. Uptime is 2h 15m 17s 300ms. job_pool-1-thread-166 State: BLOCKED CPU usage on sample: 9ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:292 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:253 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:440 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:218 sun.reflect.GeneratedMethodAccessor6.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:102 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:102 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-21 02:39:36 PM. Uptime is 2h 15m 17s 300ms. job_pool-1-thread-188 State: BLOCKED CPU usage on sample: 4ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:292 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:253 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:440 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:218 sun.reflect.GeneratedMethodAccessor6.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:102 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:102 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Potential deadlock: frozen threads found It seems that the following threads have not changed their stack for more than 10 seconds. These threads are possibly (but not necessarily!) in a deadlock or hung. BusyThread CrawlQueues.coreCrawlJob <--- Frozen for at least 14s net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:344 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:426 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Upped crawl to 100 ppm on site. Stacks at 2017-10-21 02:45:09 PM. Uptime is 2h 20m 49s 500ms. job_pool-1-thread-189 State: BLOCKED CPU usage on sample: 28ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:291 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:253 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:440 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:218 sun.reflect.GeneratedMethodAccessor6.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:102 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:102 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-21 02:45:09 PM. Uptime is 2h 20m 49s 500ms. job_pool-1-thread-166 State: BLOCKED CPU usage on sample: 24ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:291 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:253 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:440 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:218 sun.reflect.GeneratedMethodAccessor6.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:102 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:102 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-21 02:45:09 PM. Uptime is 2h 20m 49s 500ms. job_pool-1-thread-187 State: BLOCKED CPU usage on sample: 26ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:291 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:253 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:440 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:218 sun.reflect.GeneratedMethodAccessor6.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:102 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:102 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-21 02:45:51 PM. Uptime is 2h 21m 31s 500ms. job_pool-1-thread-189 State: WAITING CPU usage on sample: 4ms sun.misc.Unsafe.park(boolean, long) Unsafe.java (native) java.util.concurrent.locks.LockSupport.park(Object) LockSupport.java:175 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() AbstractQueuedSynchronizer.java:2039 java.util.concurrent.LinkedBlockingQueue.take() LinkedBlockingQueue.java:442 net.yacy.kelondro.workflow.WorkflowProcessor.take() WorkflowProcessor.java:226 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:97 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-21 02:47:03 PM. Uptime is 2h 22m 43s 600ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 1ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:47:25 PM. Uptime is 2h 23m 5s 600ms. job_pool-1-thread-188 State: BLOCKED CPU usage on sample: 22ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:291 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:253 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:440 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:218 sun.reflect.GeneratedMethodAccessor6.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:102 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:102 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-21 02:48:34 PM. Uptime is 2h 24m 14s 400ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 0ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:49:57 PM. Uptime is 2h 25m 37s 600ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 3ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Upped to 200 PPM Stacks at 2017-10-21 02:54:23 PM. Uptime is 2h 30m 3s 400ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 2ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:54:58 PM. Uptime is 2h 30m 39s 300ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 7ms net.yacy.kelondro.index.BufferedObjectIndex.size() BufferedObjectIndex.java:284 net.yacy.crawler.HostQueue.size() HostQueue.java:391 net.yacy.crawler.HostBalancer.size() HostBalancer.java:258 net.yacy.crawler.data.NoticedURL.stackSize(NoticedURL$StackType) NoticedURL.java:231 net.yacy.crawler.data.CrawlQueues.coreCrawlJobSize() CrawlQueues.java:371 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:376 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:55:15 PM. Uptime is 2h 30m 55s 600ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 5ms net.yacy.kelondro.index.BufferedObjectIndex.size() BufferedObjectIndex.java:284 net.yacy.crawler.HostQueue.size() HostQueue.java:391 net.yacy.crawler.HostBalancer.size() HostBalancer.java:258 net.yacy.crawler.data.NoticedURL.stackSize(NoticedURL$StackType) NoticedURL.java:231 net.yacy.crawler.data.CrawlQueues.coreCrawlJobSize() CrawlQueues.java:371 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:376 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:55:31 PM. Uptime is 2h 31m 11s 500ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 2ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:55:56 PM. Uptime is 2h 31m 36s 600ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 8ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:55:57 PM. Uptime is 2h 31m 37s 600ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 0ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:56:09 PM. Uptime is 2h 31m 49s 700ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 9ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:56:10 PM. Uptime is 2h 31m 50s 600ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 0ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 upped to 500 PPM running about 250 Stacks at 2017-10-21 02:58:00 PM. Uptime is 2h 33m 40s 800ms. job_pool-1-thread-189 State: BLOCKED CPU usage on sample: 2ms net.yacy.crawler.HostBalancer.push(Request, CrawlProfile, RobotsTxt) HostBalancer.java:291 net.yacy.crawler.data.NoticedURL.push(NoticedURL$StackType, Request, CrawlProfile, RobotsTxt) NoticedURL.java:253 net.yacy.crawler.CrawlStacker.stackCrawl(Request) CrawlStacker.java:440 net.yacy.crawler.CrawlStacker.job(Request) CrawlStacker.java:218 sun.reflect.GeneratedMethodAccessor6.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) DelegatingMethodAccessorImpl.java:43 java.lang.reflect.Method.invoke(Object, Object[]) Method.java:498 net.yacy.kelondro.workflow.InstantBlockingThread.job(WorkflowJob) InstantBlockingThread.java:102 net.yacy.kelondro.workflow.AbstractBlockingThread.run() AbstractBlockingThread.java:102 java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511 java.util.concurrent.FutureTask.run() FutureTask.java:266 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1149 java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:624 java.lang.Thread.run() Thread.java:748 Stacks at 2017-10-21 02:58:00 PM. Uptime is 2h 33m 40s 800ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 17ms net.yacy.kelondro.index.BufferedObjectIndex.size() BufferedObjectIndex.java:284 net.yacy.crawler.HostQueue.size() HostQueue.java:391 net.yacy.crawler.HostBalancer.size() HostBalancer.java:258 net.yacy.crawler.data.NoticedURL.stackSize(NoticedURL$StackType) NoticedURL.java:231 net.yacy.crawler.data.CrawlQueues.coreCrawlJobSize() CrawlQueues.java:371 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:376 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:58:45 PM. Uptime is 2h 34m 25s 700ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 5ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:59:04 PM. Uptime is 2h 34m 44s 800ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 11ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 Stacks at 2017-10-21 02:59:29 PM. Uptime is 2h 35m 9s 800ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 13ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 changed limit 1000 PPM running ~300 6 in a row Stacks at 2017-10-21 03:00:46 PM. Uptime is 2h 36m 26s 400ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 25ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 6000 PPM doing 25 Stacks at 2017-10-21 03:03:23 PM. Uptime is 2h 39m 4s 0ms. BusyThread CrawlQueues.coreCrawlJob [DAEMON] State: BLOCKED CPU usage on sample: 3ms net.yacy.crawler.robots.RobotsTxt.getEntry(String, ClientIdentification$Agent, boolean) RobotsTxt.java:203 net.yacy.crawler.robots.RobotsTxt.getEntry(MultiProtocolURL, ClientIdentification$Agent) RobotsTxt.java:172 net.yacy.crawler.data.Latency.waitingRobots(MultiProtocolURL, RobotsTxt, ClientIdentification$Agent) Latency.java:146 net.yacy.crawler.data.Latency.waitingRemaining(DigestURL, RobotsTxt, ClientIdentification$Agent) Latency.java:203 net.yacy.crawler.data.Latency.getDomainSleepTime(RobotsTxt, CrawlProfile, DigestURL) Latency.java:250 net.yacy.crawler.HostQueue.pop(boolean, CrawlSwitchboard, RobotsTxt) HostQueue.java:448 net.yacy.crawler.HostBalancer.pop(boolean, CrawlSwitchboard, RobotsTxt) HostBalancer.java:439 net.yacy.crawler.data.NoticedURL.pop(Balancer, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:390 net.yacy.crawler.data.NoticedURL.pop(NoticedURL$StackType, boolean, CrawlSwitchboard, RobotsTxt) NoticedURL.java:347 net.yacy.crawler.data.CrawlQueues.coreCrawlJob() CrawlQueues.java:408 net.yacy.search.Switchboard$9.jobImpl() Switchboard.java:1069 net.yacy.kelondro.workflow.InstantBusyThread.job() InstantBusyThread.java:266 net.yacy.kelondro.workflow.AbstractBusyThread.run() AbstractBusyThread.java:373 | ||||||||||||
![]() |
|
(0001485) smokingwheels (reporter) 2017-10-09 10:36 edited on: 2017-10-22 08:45 |
Maybe of sum use because of all the code is shifted a little on each file. hprof5.tar.gz 283 mb site is down |
(0001486) smokingwheels (reporter) 2017-10-09 20:04 |
yacyyourkit9-sep-17.tar.gz just some more blocked threads. Yacy is still working ok. |
(0001490) smokingwheels (reporter) 2017-10-22 08:44 |
I upgraded to version 1.921/9431 and there seems to be a lot less blocked threads. See report att. The Demo Java profiler license runs out very soon (YourKit). If you need my source I can arrange that I know the line numbers wont match the copy on github. |
![]() |
|||
Date Modified | Username | Field | Change |
2017-10-09 07:43 | smokingwheels | New Issue | |
2017-10-09 07:43 | smokingwheels | File Added: yourkit.txt | |
2017-10-09 07:44 | smokingwheels | File Added: hs_err_pid3909.log | |
2017-10-09 10:36 | smokingwheels | Note Added: 0001485 | |
2017-10-09 20:03 | smokingwheels | File Added: yacyyourkit9-sep-17.tar.gz | |
2017-10-09 20:04 | smokingwheels | Note Added: 0001486 | |
2017-10-22 08:38 | smokingwheels | File Added: report | |
2017-10-22 08:44 | smokingwheels | Note Added: 0001490 | |
2017-10-22 08:45 | smokingwheels | Note Edited: 0001485 | View Revisions |
Copyright © 2000 - 2021 MantisBT Team |