Class | Description |
---|---|
Fetcher |
A queue-based fetcher.
|
Fetcher.InputFormat | |
FetcherOutputFormat |
Splits FetcherOutput entries into multiple map files.
|
FetcherThread |
This class picks items from queues and fetches the pages.
|
FetchItem |
This class described the item to be fetched.
|
FetchItemQueue |
This class handles FetchItems which come from the same host ID (be it a
proto/hostname or proto/IP pair).
|
FetchItemQueues |
Convenience class - a collection of queues that keeps track of the total
number of items, and provides items eligible for fetching from any queue.
|
FetchNode | |
FetchNodeDb | |
QueueFeeder |
This class feeds the queues with input items, and re-fills them as items
are consumed by FetcherThread-s.
|
Copyright © 2015 The Apache Software Foundation