public class WebGraph extends Configured implements Tool
Modifier and Type | Class and Description |
---|---|
static class |
WebGraph.OutlinkDb
The OutlinkDb creates a database of all outlinks.
|
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
INLINK_DIR |
static java.lang.String |
LOCK_NAME |
static java.lang.String |
NODE_DIR |
static java.lang.String |
OLD_OUTLINK_DIR |
static java.lang.String |
OUTLINK_DIR |
Constructor and Description |
---|
WebGraph() |
Modifier and Type | Method and Description |
---|---|
void |
createWebGraph(Path webGraphDb,
Path[] segments,
boolean normalize,
boolean filter)
Creates the three different WebGraph databases, Outlinks, Inlinks, and
Node.
|
static void |
main(java.lang.String[] args) |
int |
run(java.lang.String[] args)
Parses command link arguments and runs the WebGraph jobs.
|
getConf, setConf
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getConf, setConf
public static final java.lang.String LOCK_NAME
public static final java.lang.String INLINK_DIR
public static final java.lang.String OUTLINK_DIR
public static final java.lang.String OLD_OUTLINK_DIR
public static final java.lang.String NODE_DIR
public void createWebGraph(Path webGraphDb, Path[] segments, boolean normalize, boolean filter) throws java.io.IOException, java.lang.InterruptedException, java.lang.ClassNotFoundException
webGraphDb
- The WebGraph to create or update.segments
- The array of segments used to update the WebGraph. Newer segments
and fetch times will overwrite older segments.normalize
- whether to use URLNormalizers on URL's in the segmentfilter
- whether to use URLFilters on URL's in the segmentjava.io.IOException
- If an error occurs while processing the WebGraph.java.lang.InterruptedException
java.lang.ClassNotFoundException
public static void main(java.lang.String[] args) throws java.lang.Exception
java.lang.Exception
Copyright © 2018 The Apache Software Foundation