Package org.apache.nutch.parse.js

Class JSParseFilter

java.lang.Object

org.apache.nutch.parse.js.JSParseFilter

All Implemented Interfaces:

Configurable, HtmlParseFilter, Parser, Pluggable

public class JSParseFilter
extends Object
implements HtmlParseFilter, Parser

This class is a heuristic link extractor for JavaScript files and code snippets. The general idea of a two-pass regex matching comes from Heritrix. Parts of the code come from OutlinkExtractor.java

Field Summary

Fields inherited from interface org.apache.nutch.parse.HtmlParseFilter

X_POINT_ID

Fields inherited from interface org.apache.nutch.parse.Parser

X_POINT_ID

Constructor Summary

Constructors

Constructor	Description
JSParseFilter()

Method Summary

Modifier and Type	Method	Description
ParseResult	filter(Content content, ParseResult parseResult, HTMLMetaTags metaTags, DocumentFragment doc)	Scan the JavaScript fragments of a HTML page looking for possible Outlink's
Configuration	getConf()
ParseResult	getParse(Content c)	Parse a JavaScript file and extract outlinks
static void	main(String[] args)	Main method which can be run from command line with the plugin option.
void	setConf(Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

JSParseFilter

public JSParseFilter()

Method Details

filter

public ParseResult filter(Content content,
 ParseResult parseResult,
 HTMLMetaTags metaTags,
 DocumentFragment doc)

Scan the JavaScript fragments of a HTML page looking for possible Outlink's

Specified by:

filter in interface HtmlParseFilter

Parameters:

content - page content

parseResult - parsed content, result of running the HTML parser

metaTags - within the HTMLMetaTags

doc - The DocumentFragment object

Returns:

parse the actual ParseResult object with additional outlinks from JavaScript

See Also:

• Parser.getParse(Content)

getParse

public ParseResult getParse(Content c)

Parse a JavaScript file and extract outlinks

Specified by:

getParse in interface Parser

Parameters:

c - page content

Returns:

parse the actual Parse object

main

public static void main(String[] args)
                 throws Exception

Main method which can be run from command line with the plugin option. The method takes two arguments e.g. o.a.n.parse.js.JSParseFilter file.js baseURL

Parameters:

args - run with no args to get help

Throws:

Exception - if there is a fatal error running the class with the given input

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable