0000682: Internal Index Export YAcy 1.9

(0001290)
BuBu (developer)
2016-08-30 00:44
edited on: 2016-08-30 00:56

That's likely nothing that can be fixed in the code base and a Solr index issue.
Since introduction of docvalues in the YaCy Solr schema (some time ago) https://github.com/yacy/yacy_search_server/commit/87e4abe393d70165e3d7dfaec5be7365ff2ac965#diff-0aebdb8a8b7e1ed37f8db6f5388627cf, [^] documents indexed before the change may produce this error message.
Solr docvalue references are created at index time, so the docs need to be reindexed to comply with the current sturcture.
Alternatively the doc's creating the error could be deleted.

(0001293)
LA_FORGE (reporter)
2016-09-09 19:28

Thx for the relation to my issue. So a re-crawl of all index documents utilizing the new implemented feature at the page 'IndexReIndexMonitor_p.html' will fix this issue?

(0001294)
Guims (reporter)
2016-09-15 14:41

I have 23 millions urls in my index from 1200 websites, we can't reindex all urls. What is this solution ?" Alternatively the doc's creating the error could be deleted."

(0001348)
DNcrawler (reporter)
2016-11-30 04:56

I also have the same error appearing in my logs on startup of yacy, not just during export. I have 19 million docs from 45000 sites indexed from prior versions, so if there is a way to re-index, I could do that, or just continue to ignore it hoping nothing bad is occurring within the indexes.

(0001399)
luc (reporter)
2017-04-28 11:43

A fix is proposed here https://github.com/yacy/yacy_search_server/commit/9697209ef6f672543c01e4edf2173ed90b0412fe [^]
Tested with (small) DATA and index migrated from YaCy 1.82 to latest 1.92/9185.

(0001402)
BuBu (developer)
2017-04-29 21:26

https://github.com/yacy/yacy_search_server/commit/9697209ef6f672543c01e4edf2173ed90b0412fe [^]

Issue History
Date Modified	Username	Field	Change
2016-08-29 10:41	Guims	New Issue
2016-08-30 00:44	BuBu	Note Added: 0001290
2016-08-30 00:47	BuBu	Relationship added	related to 0000681
2016-08-30 00:50	BuBu	Note Edited: 0001290	View Revisions
2016-08-30 00:56	BuBu	Note Edited: 0001290	View Revisions
2016-09-09 19:28	LA_FORGE	Note Added: 0001293
2016-09-15 14:41	Guims	Note Added: 0001294
2016-11-30 04:56	DNcrawler	Note Added: 0001348
2017-04-28 11:43	luc	Note Added: 0001399
2017-04-29 21:26	BuBu	Note Added: 0001402
2017-04-29 21:26	BuBu	Status	new => resolved
2017-04-29 21:26	BuBu	Resolution	open => fixed
2017-04-29 21:26	BuBu	Assigned To	=> administrator

Notes
(0001290) BuBu (developer) 2016-08-30 00:44 edited on: 2016-08-30 00:56	That's likely nothing that can be fixed in the code base and a Solr index issue. Since introduction of docvalues in the YaCy Solr schema (some time ago) https://github.com/yacy/yacy_search_server/commit/87e4abe393d70165e3d7dfaec5be7365ff2ac965#diff-0aebdb8a8b7e1ed37f8db6f5388627cf, [^] documents indexed before the change may produce this error message. Solr docvalue references are created at index time, so the docs need to be reindexed to comply with the current sturcture. Alternatively the doc's creating the error could be deleted.

(0001293) LA_FORGE (reporter) 2016-09-09 19:28	Thx for the relation to my issue. So a re-crawl of all index documents utilizing the new implemented feature at the page 'IndexReIndexMonitor_p.html' will fix this issue?

(0001294) Guims (reporter) 2016-09-15 14:41	I have 23 millions urls in my index from 1200 websites, we can't reindex all urls. What is this solution ?" Alternatively the doc's creating the error could be deleted."

(0001348) DNcrawler (reporter) 2016-11-30 04:56	I also have the same error appearing in my logs on startup of yacy, not just during export. I have 19 million docs from 45000 sites indexed from prior versions, so if there is a way to re-index, I could do that, or just continue to ignore it hoping nothing bad is occurring within the indexes.

(0001399) luc (reporter) 2017-04-28 11:43	A fix is proposed here https://github.com/yacy/yacy_search_server/commit/9697209ef6f672543c01e4edf2173ed90b0412fe [^] Tested with (small) DATA and index migrated from YaCy 1.82 to latest 1.92/9185.

(0001402) BuBu (developer) 2017-04-29 21:26	https://github.com/yacy/yacy_search_server/commit/9697209ef6f672543c01e4edf2173ed90b0412fe [^]