|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.conf.Configuration
org.apache.hadoop.mapred.JobConf
public class JobConf
A map/reduce job configuration. This names the Mapper
, combiner
(if any), Partitioner
, Reducer
, InputFormat
, and
OutputFormat
implementations to be used. It also indicates the set
of input files, and where the output files should be written.
Constructor Summary | |
---|---|
JobConf()
Construct a map/reduce job configuration. |
|
JobConf(Class exampleClass)
Construct a map/reduce job configuration. |
|
JobConf(Configuration conf)
Construct a map/reduce job configuration. |
|
JobConf(Configuration conf,
Class exampleClass)
Construct a map/reduce job configuration. |
|
JobConf(Path config)
Construct a map/reduce configuration. |
|
JobConf(String config)
Construct a map/reduce configuration. |
Method Summary | |
---|---|
void |
addInputPath(Path dir)
|
void |
deleteLocalFiles()
|
void |
deleteLocalFiles(String subdir)
|
Class |
getCombinerClass()
|
boolean |
getCompressMapOutput()
Are the outputs of the maps be compressed? |
InputFormat |
getInputFormat()
|
Class |
getInputKeyClass()
Deprecated. Call RecordReader.createKey() . |
Path[] |
getInputPaths()
|
Class |
getInputValueClass()
Deprecated. Call RecordReader.createValue() . |
String |
getJar()
|
String |
getJobName()
Get the user-specified job name. |
boolean |
getKeepFailedTaskFiles()
Should the temporary files for failed tasks be kept? |
String |
getKeepTaskFilesPattern()
Get the regular expression that is matched against the task names to see if we need to keep the files. |
String[] |
getLocalDirs()
|
Path |
getLocalPath(String pathString)
Constructs a local file name. |
SequenceFile.CompressionType |
getMapOutputCompressionType()
Get the compression type for the map outputs. |
Class |
getMapOutputCompressorClass(Class defaultValue)
Get the codec for compressing the map outputs |
Class |
getMapOutputKeyClass()
Get the key class for the map output data. |
Class |
getMapOutputValueClass()
Get the value class for the map output data. |
Class |
getMapperClass()
|
Class |
getMapRunnerClass()
|
int |
getMaxTaskFailuresPerTracker()
Get the maximum no. |
int |
getNumMapTasks()
|
int |
getNumReduceTasks()
|
OutputFormat |
getOutputFormat()
|
Class |
getOutputKeyClass()
|
WritableComparator |
getOutputKeyComparator()
|
Path |
getOutputPath()
|
Class |
getOutputValueClass()
|
Class |
getPartitionerClass()
|
Class |
getReducerClass()
|
boolean |
getSpeculativeExecution()
Should speculative execution be used for this job? |
Path |
getSystemDir()
|
String |
getUser()
Get the reported username for this job. |
Path |
getWorkingDirectory()
Get the current working directory for the default file system. |
void |
setCombinerClass(Class theClass)
|
void |
setCompressMapOutput(boolean compress)
Should the map outputs be compressed before transfer? Uses the SequenceFile compression. |
void |
setInputFormat(Class theClass)
|
void |
setInputKeyClass(Class theClass)
Deprecated. Not used |
void |
setInputPath(Path dir)
|
void |
setInputValueClass(Class theClass)
Deprecated. Not used |
void |
setJar(String jar)
|
void |
setJarByClass(Class cls)
Set the job's jar file by finding an example class location. |
void |
setJobName(String name)
Set the user-specified job name. |
void |
setKeepFailedTaskFiles(boolean keep)
Set whether the framework should keep the intermediate files for failed tasks. |
void |
setKeepTaskFilesPattern(String pattern)
Set a regular expression for task names that should be kept. |
void |
setMapOutputCompressionType(SequenceFile.CompressionType style)
Set the compression type for the map outputs. |
void |
setMapOutputCompressorClass(Class codecClass)
Set the given class as the compression codec for the map outputs. |
void |
setMapOutputKeyClass(Class theClass)
Set the key class for the map output data. |
void |
setMapOutputValueClass(Class theClass)
Set the value class for the map output data. |
void |
setMapperClass(Class theClass)
|
void |
setMapRunnerClass(Class theClass)
|
void |
setMaxTaskFailuresPerTracker(int noFailures)
Set the maximum no. |
void |
setNumMapTasks(int n)
|
void |
setNumReduceTasks(int n)
|
void |
setOutputFormat(Class theClass)
|
void |
setOutputKeyClass(Class theClass)
|
void |
setOutputKeyComparatorClass(Class theClass)
|
void |
setOutputPath(Path dir)
|
void |
setOutputValueClass(Class theClass)
|
void |
setPartitionerClass(Class theClass)
|
void |
setReducerClass(Class theClass)
|
void |
setSpeculativeExecution(boolean new_val)
Turn on or off speculative execution for this job. |
void |
setUser(String user)
Set the reported username for this job. |
void |
setWorkingDirectory(Path dir)
Set the current working directory for the default file system |
Methods inherited from class org.apache.hadoop.conf.Configuration |
---|
addDefaultResource, addDefaultResource, addDefaultResource, addFinalResource, addFinalResource, addFinalResource, entries, get, get, get, getBoolean, getClass, getClass, getClassByName, getClassLoader, getConfResourceAsInputStream, getConfResourceAsReader, getFile, getFloat, getInt, getLocalPath, getLong, getObject, getResource, getStrings, main, set, setBoolean, setClass, setClassLoader, setInt, setLong, setObject, setQuietMode, toString, write |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Constructor Detail |
---|
public JobConf()
public JobConf(Class exampleClass)
exampleClass
- a class whose containing jar is used as the job's jar.public JobConf(Configuration conf)
conf
- a Configuration whose settings will be inherited.public JobConf(Configuration conf, Class exampleClass)
conf
- a Configuration whose settings will be inherited.exampleClass
- a class whose containing jar is used as the job's jar.public JobConf(String config)
config
- a Configuration-format XML job description filepublic JobConf(Path config)
config
- a Configuration-format XML job description fileMethod Detail |
---|
public String getJar()
public void setJar(String jar)
public void setJarByClass(Class cls)
cls
- the example classpublic Path getSystemDir()
public String[] getLocalDirs() throws IOException
IOException
public void deleteLocalFiles() throws IOException
IOException
public void deleteLocalFiles(String subdir) throws IOException
IOException
public Path getLocalPath(String pathString) throws IOException
IOException
public void setInputPath(Path dir)
public void addInputPath(Path dir)
public Path[] getInputPaths()
public String getUser()
public void setUser(String user)
user
- the usernamepublic void setKeepFailedTaskFiles(boolean keep)
public boolean getKeepFailedTaskFiles()
public void setKeepTaskFilesPattern(String pattern)
pattern
- the java.util.regex.Pattern to match against the
task names.public String getKeepTaskFilesPattern()
public void setWorkingDirectory(Path dir)
dir
- the new current working directorypublic Path getWorkingDirectory()
public Path getOutputPath()
public void setOutputPath(Path dir)
public InputFormat getInputFormat()
public void setInputFormat(Class theClass)
public OutputFormat getOutputFormat()
public void setOutputFormat(Class theClass)
public Class getInputKeyClass()
RecordReader.createKey()
.
public void setInputKeyClass(Class theClass)
public Class getInputValueClass()
RecordReader.createValue()
.
public void setInputValueClass(Class theClass)
public void setCompressMapOutput(boolean compress)
public boolean getCompressMapOutput()
public void setMapOutputCompressionType(SequenceFile.CompressionType style)
style
- NONE, RECORD, or BLOCK to control how the map outputs are
compressedpublic SequenceFile.CompressionType getMapOutputCompressionType()
public void setMapOutputCompressorClass(Class codecClass)
codecClass
- the CompressionCodec class that will compress the
map outputspublic Class getMapOutputCompressorClass(Class defaultValue)
defaultValue
- the value to return if it is not set
IllegalArgumentException
- if the class was specified, but not foundpublic Class getMapOutputKeyClass()
public void setMapOutputKeyClass(Class theClass)
public Class getMapOutputValueClass()
public void setMapOutputValueClass(Class theClass)
public Class getOutputKeyClass()
public void setOutputKeyClass(Class theClass)
public WritableComparator getOutputKeyComparator()
public void setOutputKeyComparatorClass(Class theClass)
public Class getOutputValueClass()
public void setOutputValueClass(Class theClass)
public Class getMapperClass()
public void setMapperClass(Class theClass)
public Class getMapRunnerClass()
public void setMapRunnerClass(Class theClass)
public Class getPartitionerClass()
public void setPartitionerClass(Class theClass)
public Class getReducerClass()
public void setReducerClass(Class theClass)
public Class getCombinerClass()
public void setCombinerClass(Class theClass)
public boolean getSpeculativeExecution()
public void setSpeculativeExecution(boolean new_val)
public int getNumMapTasks()
public void setNumMapTasks(int n)
public int getNumReduceTasks()
public void setNumReduceTasks(int n)
public String getJobName()
public void setJobName(String name)
name
- the job's new namepublic void setMaxTaskFailuresPerTracker(int noFailures)
noFailures
- maximum no. of failures of a given job per tasktracker.public int getMaxTaskFailuresPerTracker()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |