YaCy-Bugtracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000624YaCy[All Projects] Generalpublic2015-12-07 09:362015-12-07 09:36
Reportersmokingwheels 
Assigned To 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusnewResolutionopen 
ETAnone 
Platformx86OSWindowsOS Version2003
Product VersionYaCy 1.8 
Target VersionFixed in Version 
Summary0000624: java.lang.NullPointerException when crawing a file with 10k URL's
DescriptionCrawl a file with lots of URL's in Advanced Crawler depth 1.
I downgraded to 1.8/9000 and crawling worked ok.

I have Git and Ant installed so I cloned yacy_search_server and built it, now I can crawl the urls ok.

Also building the windows version of yacy fails.
Additional InformationI 2015/12/06 17:31:04 STACKCRAWL LOST STACKER PROFILE HANDLE 'L-TQG-FEUymn' for URL https://twitter.com/Juniorbvm [^]
I 2015/12/06 17:31:04 REJECTED https://twitter.com/Juniorbvm [^] - LOST STACKER PROFILE HANDLE 'L-TQG-FEUymn' for URL https://twitter.com/Juniorbvm [^]


W 2015/12/06 17:52:37 ConcurrentLog java.lang.NullPointerException
java.lang.NullPointerException
    at net.yacy.cora.order.NaturalOrder.compare(NaturalOrder.java:148)
    at net.yacy.kelondro.index.RAMIndexCluster$1.compare(RAMIndexCluster.java:339)
    at net.yacy.kelondro.index.RAMIndexCluster$1.compare(RAMIndexCluster.java:336)
    at net.yacy.kelondro.util.StackIterator.next(StackIterator.java:91)
    at net.yacy.kelondro.index.RowHandleMap$2.next(RowHandleMap.java:501)
    at net.yacy.kelondro.index.RowHandleMap$2.next(RowHandleMap.java:492)
    at net.yacy.kelondro.table.Table.<init>(Table.java:213)
    at net.yacy.crawler.HostQueue.openStack(HostQueue.java:234)
    at net.yacy.crawler.HostQueue.openAllStacks(HostQueue.java:140)
    at net.yacy.crawler.HostQueue.init(HostQueue.java:121)
    at net.yacy.crawler.HostQueue.<init>(HostQueue.java:86)
    at net.yacy.crawler.HostBalancer.push(HostBalancer.java:249)
    at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:184)
    at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:355)
    at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
    at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
    at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)


Some of the errors when trying to build yacy the windows version not sure if it matters?
: error: cannot find symbol
    [javac] if (osc.init(name, url)) {
    [javac] ^
    [javac] symbol: method init(String,String)
    [javac] location: variable osc of type OpenSearchConnector
    [javac] F:\YaCy\source\net\yacy\cora\federate\FederateSearchManager.java:11
: error: no suitable method found for add(OpenSearchConnector)
    [javac] conlist.add(osc);
    [javac] ^
    [javac] method Collection.add(AbstractFederateSearchConnector) is not a
plicable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method Set.add(AbstractFederateSearchConnector) is not applicab
e
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method AbstractCollection.add(AbstractFederateSearchConnector)
s not applicable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method HashSet.add(AbstractFederateSearchConnector) is not appl
cable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] F:\YaCy\source\net\yacy\cora\federate\FederateSearchManager.java:18
: error: constructor QueryModifier in class QueryModifier cannot be applied to
iven types;
    [javac] new QueryModifier(0),
    [javac] ^
    [javac] required: no arguments
    [javac] found: int
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\cora\federate\FederateSearchManager.java:23
: error: constructor OpenSearchConnector in class OpenSearchConnector cannot be
applied to given types;
    [javac] OpenSearchConnector osd = new OpenSearchCon
ector();
    [javac] ^
    [javac] required: Switchboard,boolean
    [javac] found: no arguments
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\cora\federate\FederateSearchManager.java:24
: error: cannot find symbol
    [javac] if (osd.init(name, urlTemplate)) {
    [javac] ^
    [javac] symbol: method init(String,String)
    [javac] location: variable osd of type OpenSearchConnector
    [javac] F:\YaCy\source\net\yacy\cora\federate\FederateSearchManager.java:24
: error: no suitable method found for add(OpenSearchConnector)
    [javac] conlist.add(osd);
    [javac] ^
    [javac] method Collection.add(AbstractFederateSearchConnector) is not a
plicable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method Set.add(AbstractFederateSearchConnector) is not applicab
e
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method AbstractCollection.add(AbstractFederateSearchConnector)
s not applicable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method HashSet.add(AbstractFederateSearchConnector) is not appl
cable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] F:\YaCy\source\net\yacy\cora\federate\FederateSearchManager.java:41
: error: constructor OpenSearchConnector in class OpenSearchConnector cannot be
applied to given types;
    [javac] osd = new OpenSearchConnector();
    [javac] ^
    [javac] required: Switchboard,boolean
    [javac] found: no arguments
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\cora\federate\FederateSearchManager.java:41
: error: cannot find symbol
    [javac] if (osd.init(name, url)) {
    [javac] ^
    [javac] symbol: method init(String,String)
    [javac] location: variable osd of type OpenSearchConnector
    [javac] F:\YaCy\source\net\yacy\cora\federate\FederateSearchManager.java:41
: error: no suitable method found for add(OpenSearchConnector)
    [javac] conlist.add(osd);
    [javac] ^
    [javac] method Collection.add(AbstractFederateSearchConnector) is not a
plicable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method Set.add(AbstractFederateSearchConnector) is not applicab
e
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method AbstractCollection.add(AbstractFederateSearchConnector)
s not applicable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] method HashSet.add(AbstractFederateSearchConnector) is not appl
cable
    [javac] (argument mismatch; OpenSearchConnector cannot be converted t
 AbstractFederateSearchConnector)
    [javac] F:\YaCy\source\net\yacy\cora\federate\solr\logic\BooleanLiteral.jav
:33: error: constructor Literal in class Literal cannot be applied to given typ
s;
    [javac] super();
    [javac] ^
    [javac] required: SchemaDeclaration,String
    [javac] found: no arguments
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\cora\federate\solr\logic\CatchallLiteral.ja
a:33: error: constructor Literal in class Literal cannot be applied to given ty
es;
    [javac] super();
    [javac] ^
    [javac] required: SchemaDeclaration,String
    [javac] found: no arguments
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\cora\federate\solr\logic\LongLiteral.java:3
: error: constructor Literal in class Literal cannot be applied to given types;
    [javac] super();
    [javac] ^
    [javac] required: SchemaDeclaration,String
    [javac] found: no arguments
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\cora\federate\solr\logic\StringLiteral.java
34: error: constructor Literal in class Literal cannot be applied to given type
;
    [javac] super();
    [javac] ^
    [javac] required: SchemaDeclaration,String
    [javac] found: no arguments
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\cora\util\Html2Image.java:168: error: incom
atible types: String[] cannot be converted to String
    [javac] message = OS.execSynchronous(commandx);
    [javac] ^
    [javac] F:\YaCy\source\net\yacy\crawler\RecrawlBusyThread.java:64: error: c
nstructor AbstractBusyThread in class AbstractBusyThread cannot be applied to g
ven types;
    [javac] super(3000, 1000); // set lower limits of cycle delay
    [javac] ^
    [javac] required: long,long,long,long
    [javac] found: int,int
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\crawler\data\Snapshots.java:235: error: can
ot find symbol
    [javac] String ds = GenericFormatter.SHORT_MINUTE_FORMATTER.format(
ate);
    [javac] ^
    [javac] symbol: variable SHORT_MINUTE_FORMATTER
    [javac] location: class GenericFormatter
    [javac] F:\YaCy\source\net\yacy\crawler\data\Snapshots.java:249: error: can
ot find symbol
    [javac] String ds = GenericFormatter.SHORT_MINUTE_FORMATTER.format(
ate);
    [javac] ^
    [javac] symbol: variable SHORT_MINUTE_FORMATTER
    [javac] location: class GenericFormatter
    [javac] F:\YaCy\source\net\yacy\crawler\data\Snapshots.java:362: error: can
ot find symbol
    [javac] return GenericFormatter.SHORT_MINUTE_FORMATTER.parse(d,
0).getTime();
    [javac] ^
    [javac] symbol: variable SHORT_MINUTE_FORMATTER
    [javac] location: class GenericFormatter
    [javac] F:\YaCy\source\net\yacy\crawler\data\Snapshots.java:365: error: inc
mpatible types: int cannot be converted to String
    [javac] return GenericFormatter.SHORT_DAY_FORMATTER.parse(d
 0).getTime();
    [javac]
 ^
    [javac] F:\YaCy\source\net\yacy\crawler\data\Transactions.java:175: error:
ethod writeDoc in class EnhancedXMLResponseWriter cannot be applied to given ty
es;
    [javac] EnhancedXMLResponseWriter.writeDoc(osw, doc);
    [javac] ^
    [javac] required: Writer,IndexSchema,String,Document,float,boolean
    [javac] found: OutputStreamWriter,SolrInputDocument
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\document\Tokenizer.java:96: error: cannot f
nd symbol
    [javac] Set<String> vocabularyNames = LibraryProvider.a
totagging.getVocabularyNames();
    [javac]
         ^
    [javac] symbol: method getVocabularyNames()
    [javac] location: variable autotagging of type AutotaggingLibrary
    [javac] F:\YaCy\source\net\yacy\document\Tokenizer.java:131: error: method
etTagFromTerm in class AutotaggingLibrary cannot be applied to given types;
    [javac] tag = LibraryProvider.autotagging.getTagFro
Term(vocabularyNames, testterm);
    [javac] ^
    [javac] required: String
    [javac] found: Set<String>,String
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\document\Tokenizer.java:212: error: non-sta
ic method size() cannot be referenced from a static context
    [javac] if (SynonymLibrary.size() > 0) {
    [javac] ^
    [javac] F:\YaCy\source\net\yacy\document\Tokenizer.java:214: error: non-sta
ic method getSynonyms(String) cannot be referenced from a static context
    [javac] Set<String> syms = SynonymLibrary.getSynonyms(word)

    [javac] ^
    [javac] F:\YaCy\source\net\yacy\document\VocabularyScraper.java:53: error:
annot find symbol
    [javac] for (String voc: this.scraperDefinition.keySet()) {
    [javac] ^
    [javac] symbol: method keySet()
    [javac] location: variable scraperDefinition of type JSONObject
    [javac] F:\YaCy\source\net\yacy\document\parser\images\metadataImageParser.
ava:65: error: metadataImageParser is not abstract and does not override abstra
t method parse(AnchorURL,String,String,InputStream) in Parser
    [javac] public class metadataImageParser extends AbstractParser implements
arser {
    [javac] ^
    [javac] F:\YaCy\source\net\yacy\document\parser\images\metadataImageParser.
ava:86: error: method does not override or implement a method from a supertype
    [javac] @Override
    [javac] ^
    [javac] F:\YaCy\source\net\yacy\document\parser\images\metadataImageParser.
ava:104: error: incompatible types: BufferedInputStream cannot be converted to
ile
    [javac] final Metadata metadata = ImageMetadataReader.readMetad
ta(new BufferedInputStream(source));
    [javac]
   ^
    [javac] F:\YaCy\source\net\yacy\document\parser\images\svgParser.java:57: e
ror: svgParser is not abstract and does not override abstract method parse(Anch
rURL,String,String,InputStream) in Parser
    [javac] public class svgParser extends AbstractParser implements Parser {
    [javac] ^
    [javac] F:\YaCy\source\net\yacy\document\parser\images\svgParser.java:80: e
ror: method does not override or implement a method from a supertype
    [javac] @Override
    [javac] ^
    [javac] F:\YaCy\source\net\yacy\search\AutoSearch.java:74: error: construct
r AbstractBusyThread in class AbstractBusyThread cannot be applied to given typ
s;
    [javac] super(3000, 1000); // set lower limits of cycle delay
    [javac] ^
    [javac] required: long,long,long,long
    [javac] found: int,int
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\search\AutoSearch.java:161: error: no suita
le method found for getBookmarksIterator(no arguments)
    [javac] Iterator<Bookmark> it = Switchboard.getSwitchboard().bookma
ksDB.getBookmarksIterator();
    [javac]
    ^
    [javac] method BookmarksDB.getBookmarksIterator(boolean) is not applica
le
    [javac] (actual and formal argument lists differ in length)
    [javac] method BookmarksDB.getBookmarksIterator(String,boolean) is not
pplicable
    [javac] (actual and formal argument lists differ in length)
    [javac] F:\YaCy\source\net\yacy\search\AutoSearch.java:169: error: cannot f
nd symbol
    [javac] final String query = bmk.getQuery();
    [javac] ^
    [javac] symbol: method getQuery()
    [javac] location: variable bmk of type BookmarksDB.Bookmark
    [javac] F:\YaCy\source\net\yacy\search\AutoSearch.java:249: error: method g
tPublicAddress in class Seed cannot be applied to given types;
    [javac] RemoteInstance instance = new RemoteInstance("http:
/" + seed.getPublicAddress(seed.getIP()) + "/solr/", null, null, 10000); // thi
 is a 'patch configuration' which considers 'solr' as default collection
    [javac]
         ^
    [javac] required: no arguments
    [javac] found: String
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\search\AutoSearch.java:275: error: method g
tPublicAddress in class Seed cannot be applied to given types;
    [javac] final String rssSearchServiceURL = "http://" [^] + seed.get
ublicAddress(seed.getIP()) + "/yacysearch.rss";
    [javac] ^
    [javac] required: no arguments
    [javac] found: String
    [javac] reason: actual and formal argument lists differ in length
    [javac] F:\YaCy\source\net\yacy\utils\translation\ListNonTranslatedFiles.ja
a:92: error: cannot find symbol
    [javac] List<File> srcFiles = FileUtils.getFilesRecursi
e(sourceDir, excludedDir, fileFilter);
    [javac] ^
    [javac] symbol: method getFilesRecursive(File,String,FilenameFilter)
    [javac] location: class FileUtils
    [javac] Note: Some messages have been simplified; recompile with -Xdiags:ve
bose to get full output
    [javac] 40 errors
    [javac] 1 warning

BUILD FAILED
F:\YaCy\build.xml:257: Compile failed; see the compiler error output for detail
.

Total time: 37 seconds

F:\YaCy>
TagsNo tags attached.
Attached Filestxt file icon 0.TXT [^] (3,207,509 bytes) 2015-12-07 09:36

- Relationships

-  Notes
There are no notes attached to this issue.

- Issue History
Date Modified Username Field Change
2015-12-07 09:36 smokingwheels New Issue
2015-12-07 09:36 smokingwheels File Added: 0.TXT


Copyright © 2000 - 2019 MantisBT Team
Powered by Mantis Bugtracker