Package org.apache.nutch.indexer

package org.apache.nutch.indexer

Index content, configure and run indexing and cleaning jobs to add, update, and delete documents from an index. Two tasks are delegated to plugins:

• indexing filters, which fill index fields of each document
• index writer plugins; which send documents to index back-ends (Solr, etc.).

Related Packages

Package	Description
org.apache.nutch.indexer.anchor	An indexing plugin for inbound anchor text.
org.apache.nutch.indexer.arbitrary	Indexing filter to add document arbitrary data to the index from the output of a user-specified class.
org.apache.nutch.indexer.basic	A basic indexing plugin, adds basic fields: url, host, title, content, etc.
org.apache.nutch.indexer.feed	Indexing filter to index meta data from RSS feeds.
org.apache.nutch.indexer.filter
org.apache.nutch.indexer.geoip	This plugin implements an indexing filter which takes advantage of the GeoIP2-java API.
org.apache.nutch.indexer.jexl	This plugin implements a dynamic indexing filter which uses JEXL expressions to allow filtering based on the page's metadata
org.apache.nutch.indexer.links
org.apache.nutch.indexer.metadata	Indexing filter to add document metadata to the index.
org.apache.nutch.indexer.more	A more indexing plugin, adds "more" index fields:last modified date, MIME type, content length.
org.apache.nutch.indexer.replace	Indexing filter to allow pattern replacements on metadata.
org.apache.nutch.indexer.staticfield	A simple plugin called at indexing that adds fields with static data.
org.apache.nutch.indexer.subcollection	Indexing filter to assign documents to subcollections.
org.apache.nutch.indexer.tld	Top Level Domain Indexing plugin.
org.apache.nutch.indexer.urlmeta	URL Meta Tag Indexing Plugin

Class	Description
CleaningJob	The class scans CrawlDB looking for entries with status DB_GONE (404) or DB_DUPLICATE and sends delete requests to indexers for those documents.
CleaningJob.DBFilter
CleaningJob.DeleterReducer
IndexerMapReduce	This class is typically invoked from within IndexingJob and handles all MapReduce functionality required when undertaking indexing.
IndexerMapReduce.IndexerMapper
IndexerMapReduce.IndexerReducer
IndexerOutputFormat
IndexingException
IndexingFilter	Extension point for indexing.
IndexingFilters	Creates and caches IndexingFilter implementing plugins.
IndexingFiltersChecker	Reads and parses a URL and run the indexers on it.
IndexingJob	Generic indexer which relies on the plugins implementing IndexWriter
IndexWriter
IndexWriterConfig
IndexWriterParams
IndexWriters	Creates and caches IndexWriter implementing plugins.
NutchDocument	A NutchDocument is the unit of indexing.
NutchField	This class represents a multi-valued field with a weight.
NutchIndexAction	A NutchIndexAction is the new unit of indexing holding the document and action information.