Package org.apache.nutch.parse.headings

Class HeadingsParseFilter

java.lang.Object

org.apache.nutch.parse.headings.HeadingsParseFilter

All Implemented Interfaces:

Configurable, HtmlParseFilter, Pluggable

public class HeadingsParseFilter
extends Object
implements HtmlParseFilter

HtmlParseFilter to retrieve h1 and h2 values from the DOM.

Field Summary

Fields

Modifier and Type	Field	Description
protected static Pattern	whitespacePattern	Pattern used to strip surpluss whitespace

Fields inherited from interface org.apache.nutch.parse.HtmlParseFilter

X_POINT_ID

Constructor Summary

Constructors

Constructor	Description
HeadingsParseFilter()

Method Summary

Modifier and Type	Method	Description
ParseResult	filter(Content content, ParseResult parseResult, HTMLMetaTags metaTags, DocumentFragment doc)	Adds metadata or otherwise modifies a parse of HTML content, given the DOM tree of a page.
Configuration	getConf()
protected List<String>	getElement(DocumentFragment doc, String element)	Finds the specified element and returns its value
protected static String	getNodeValue(Node node)	Returns the text value of the specified Node and child nodes
void	setConf(Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

whitespacePattern

protected static Pattern whitespacePattern

Pattern used to strip surpluss whitespace

Constructor Details

HeadingsParseFilter

public HeadingsParseFilter()

Method Details

filter

public ParseResult filter(Content content,
 ParseResult parseResult,
 HTMLMetaTags metaTags,
 DocumentFragment doc)

Description copied from interface: HtmlParseFilter

Adds metadata or otherwise modifies a parse of HTML content, given the DOM tree of a page.

Specified by:

filter in interface HtmlParseFilter

Parameters:

content - the Content for a given response

parseResult - the result of running on or more Parser's on the content.

metaTags - a populated HTMLMetaTags object

doc - a DocumentFragment (DOM) which can be processed in the filtering process.

Returns:

a filtered ParseResult

See Also:

• Parser.getParse(Content)

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable

getElement

protected List<String> getElement(DocumentFragment doc,
 String element)

Finds the specified element and returns its value

Parameters:

doc - the input DocumentFragment to process

element - the element to find in the DocumentFragment

Returns:

a List containing headings

getNodeValue

protected static String getNodeValue(Node node)

Returns the text value of the specified Node and child nodes

Parameters:

node - the input Node to extract a value(s) for

Returns:

the whitespace-stripped String node value(s)