org.apache.hadoop.chukwa.datacollection.adaptor.filetailer
Class FileTailingAdaptor

java.lang.Object
  extended by org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.FileTailingAdaptor
All Implemented Interfaces:
Adaptor
Direct Known Subclasses:
CharFileTailingAdaptorUTF8, CharFileTailingAdaptorUTF8NewLineEscaped

public class FileTailingAdaptor
extends Object
implements Adaptor

An adaptor that repeatedly tails a specified file, sending the new bytes. This class does not split out records, but just sends everything up to end of file. Subclasses can alter this behavior by overriding extractRecords().


Field Summary
protected  long adaptorID
           
protected static org.apache.hadoop.conf.Configuration conf
           
static int DEFAULT_MAX_READ_SIZE
          This is the maximum amount we'll read from any one file before moving on to the next.
protected  long fileReadOffset
          next PHYSICAL offset to read
static int GRACEFUL_PERIOD
           
static int MAX_READ_SIZE
           
static int MAX_RETRIES
           
protected  RandomAccessFile reader
           
protected  String type
           
 
Constructor Summary
FileTailingAdaptor()
           
 
Method Summary
protected  int extractRecords(ChunkReceiver eq, long buffOffsetInFile, byte[] buf)
          Extract records from a byte sequence
 String getCurrentStatus()
          Return the adaptor's state Should not include class name, datatype or byte offset, which are written by caller.
 String getStreamName()
          Return the stream name
 String getType()
           
 void hardStop()
          Stop tailing the file, effective immediately.
 long shutdown()
          Do one last tail, and then stop
 void start(long adaptorID, String type, String params, long bytes, ChunkReceiver dest)
          Start this adaptor
 boolean tailFile(ChunkReceiver eq)
          Looks at the tail of the associated file, adds some of it to event queue This method is not thread safe.
 String toString()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

DEFAULT_MAX_READ_SIZE

public static final int DEFAULT_MAX_READ_SIZE
This is the maximum amount we'll read from any one file before moving on to the next. This way, we get quick response time for other files if one file is growing rapidly.

See Also:
Constant Field Values

MAX_READ_SIZE

public static int MAX_READ_SIZE

MAX_RETRIES

public static int MAX_RETRIES

GRACEFUL_PERIOD

public static int GRACEFUL_PERIOD

conf

protected static org.apache.hadoop.conf.Configuration conf

fileReadOffset

protected long fileReadOffset
next PHYSICAL offset to read


type

protected String type

reader

protected RandomAccessFile reader

adaptorID

protected long adaptorID
Constructor Detail

FileTailingAdaptor

public FileTailingAdaptor()
Method Detail

start

public void start(long adaptorID,
                  String type,
                  String params,
                  long bytes,
                  ChunkReceiver dest)
Description copied from interface: Adaptor
Start this adaptor

Specified by:
start in interface Adaptor
type - the application type, who is starting this adaptor
params - the status string to use for configuration.
bytes - the stream offset of the first byte sent by this adaptor

shutdown

public long shutdown()
              throws AdaptorException
Do one last tail, and then stop

Specified by:
shutdown in interface Adaptor
Returns:
the logical offset at which the adaptor stops
Throws:
AdaptorException
See Also:
Adaptor.shutdown()

hardStop

public void hardStop()
              throws AdaptorException
Stop tailing the file, effective immediately.

Specified by:
hardStop in interface Adaptor
Throws:
AdaptorException

getCurrentStatus

public String getCurrentStatus()
Description copied from interface: Adaptor
Return the adaptor's state Should not include class name, datatype or byte offset, which are written by caller.

Specified by:
getCurrentStatus in interface Adaptor
Returns:
the adaptor state as a string
See Also:
Adaptor.getCurrentStatus()

toString

public String toString()
Overrides:
toString in class Object

getStreamName

public String getStreamName()
Description copied from interface: Adaptor
Return the stream name

Specified by:
getStreamName in interface Adaptor
Returns:
Stream name as a string

tailFile

public boolean tailFile(ChunkReceiver eq)
                 throws InterruptedException
Looks at the tail of the associated file, adds some of it to event queue This method is not thread safe. Returns true if there's more data in the file

Parameters:
eq - the queue to write Chunks into
Throws:
InterruptedException

extractRecords

protected int extractRecords(ChunkReceiver eq,
                             long buffOffsetInFile,
                             byte[] buf)
                      throws InterruptedException
Extract records from a byte sequence

Parameters:
eq - the queue to stick the new chunk[s] in
buffOffsetInFile - the byte offset in the stream at which buf[] begins
buf - the byte buffer to extract records from
Returns:
the number of bytes processed
Throws:
InterruptedException

getType

public String getType()
Specified by:
getType in interface Adaptor


Copyright © ${year} The Apache Software Foundation