Uses of Interface
org.apache.nutch.protocol.Protocol
Packages that use Protocol
Package
Description
Classes related to the
Protocol interface,
see also org.apache.nutch.net.protocols.Protocol plugin which supports retrieving local file resources.
Protocol plugin which supports retrieving documents via the ftp protocol.
Protocol plugin which supports retrieving documents via HTTP/HTTPS using
Selenium and the
HtmlUnitDriver web
driver for the for the
HtmlUnit headless browser.
Protocol plugin which supports retrieving documents via the http protocol.
Common API used by HTTP plugins (
http,
httpclient, etc.)Protocol plugin which supports retrieving documents via the HTTP andHTTPS
protocols, optionally with Basic, Digest and NTLM authentication schemes for
web server as well as proxy server.
Protocol plugin which supports retrieving documents using and interacting
with Selenium.
Protocol plugin for HTTP/HTTPS based on
okhttp, supports HTTP 1.1
and/or http/2.
Protocol plugin which supports retrieving documents via
Selenium.
-
Uses of Protocol in org.apache.nutch.protocol
Methods in org.apache.nutch.protocol that return ProtocolModifier and TypeMethodDescriptionProtocolFactory.getProtocol(String urlString) Returns the appropriateProtocolimplementation for a url.ProtocolFactory.getProtocol(URL url) Returns the appropriateProtocolimplementation for a url.ProtocolFactory.getProtocolById(String id) Methods in org.apache.nutch.protocol with parameters of type ProtocolModifier and TypeMethodDescriptionabstract crawlercommons.robots.BaseRobotRulesRobotRulesParser.getRobotRulesSet(Protocol protocol, URL url, List<Content> robotsTxtContent) Fetch robots.txt (or it's protocol-specific equivalent) which applies to the given URL, parse it and return the set of robot rules applicable for the configured agent name(s).crawlercommons.robots.BaseRobotRulesRobotRulesParser.getRobotRulesSet(Protocol protocol, Text url, List<Content> robotsTxtContent) Fetch robots.txt (or it's protocol-specific equivalent) which applies to the given URL, parse it and return the set of robot rules applicable for the configured agent name(s). -
Uses of Protocol in org.apache.nutch.protocol.file
Classes in org.apache.nutch.protocol.file that implement ProtocolModifier and TypeClassDescriptionclassThis class is a protocol plugin used for file: scheme. -
Uses of Protocol in org.apache.nutch.protocol.ftp
Classes in org.apache.nutch.protocol.ftp that implement ProtocolMethods in org.apache.nutch.protocol.ftp with parameters of type ProtocolModifier and TypeMethodDescriptioncrawlercommons.robots.BaseRobotRulesFtpRobotRulesParser.getRobotRulesSet(Protocol ftp, URL url, List<Content> robotsTxtContent) The hosts for which the caching of robots rules is yet to be done, it sends a Ftp request to the host corresponding to theURLpassed, gets robots file, parses the rules and caches the rules object to avoid re-work in future. -
Uses of Protocol in org.apache.nutch.protocol.htmlunit
Classes in org.apache.nutch.protocol.htmlunit that implement Protocol -
Uses of Protocol in org.apache.nutch.protocol.http
Classes in org.apache.nutch.protocol.http that implement Protocol -
Uses of Protocol in org.apache.nutch.protocol.http.api
Classes in org.apache.nutch.protocol.http.api that implement ProtocolMethods in org.apache.nutch.protocol.http.api with parameters of type ProtocolModifier and TypeMethodDescriptioncrawlercommons.robots.BaseRobotRulesHttpRobotRulesParser.getRobotRulesSet(Protocol http, URL url, List<Content> robotsTxtContent) Get the rules from robots.txt which applies for the givenurl. -
Uses of Protocol in org.apache.nutch.protocol.httpclient
Classes in org.apache.nutch.protocol.httpclient that implement ProtocolModifier and TypeClassDescriptionclassThis class is a protocol plugin that configures an HTTP client for Basic, Digest and NTLM authentication schemes for web server as well as proxy server. -
Uses of Protocol in org.apache.nutch.protocol.interactiveselenium
Classes in org.apache.nutch.protocol.interactiveselenium that implement Protocol -
Uses of Protocol in org.apache.nutch.protocol.okhttp
Classes in org.apache.nutch.protocol.okhttp that implement Protocol -
Uses of Protocol in org.apache.nutch.protocol.selenium
Classes in org.apache.nutch.protocol.selenium that implement Protocol