org.apache.hadoop.chukwa.datacollection.adaptor.filetailer
Class LWFTAdaptor

java.lang.Object
  extended by org.apache.hadoop.chukwa.datacollection.adaptor.AbstractAdaptor
      extended by org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.LWFTAdaptor
All Implemented Interfaces:
Adaptor
Direct Known Subclasses:
FileTailingAdaptor

public class LWFTAdaptor
extends AbstractAdaptor

A base class for file tailing adaptors. Intended to mandate as little policy as possible, and to use as few system resources as possible. If the file does not exist, this class will continue to retry quietly forever and will start tailing if it's eventually created.


Field Summary
protected  org.apache.hadoop.conf.Configuration conf
           
static int DEFAULT_MAX_READ_SIZE
          This is the maximum amount we'll read from any one file before moving on to the next.
protected  long fileReadOffset
          next PHYSICAL offset to read
static int MAX_READ_SIZE
           
static String MAX_READ_SIZE_OPT
           
protected  long offsetOfFirstByte
          The logical offset of the first byte of the file
protected static org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.FileTailer tailer
           
 
Fields inherited from class org.apache.hadoop.chukwa.datacollection.adaptor.AbstractAdaptor
adaptorID, control, dest, type
 
Constructor Summary
LWFTAdaptor()
           
 
Method Summary
protected  int extractRecords(ChunkReceiver eq, long buffOffsetInFile, byte[] buf)
          Extract records from a byte sequence
 String getCurrentStatus()
          Return the adaptor's state Should not include class name or byte offset, which are written by caller.
 String getStreamName()
           
 void hardStop()
          Signals this adaptor to come to an abrupt stop, as quickly as it can.
 String parseArgs(String params)
          Parse args, return stream name.
 long shutdown()
          Signals this adaptor to come to an orderly stop.
 long shutdown(AdaptorShutdownPolicy shutdownPolicy)
          Signals this adaptor to come to an orderly stop.
protected  boolean slurp(long len, RandomAccessFile reader)
           
 void start(long offset)
           
 boolean tailFile(ChunkReceiver eq)
           
 String toString()
           
 
Methods inherited from class org.apache.hadoop.chukwa.datacollection.adaptor.AbstractAdaptor
deregisterAndStop, getType, start
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

DEFAULT_MAX_READ_SIZE

public static final int DEFAULT_MAX_READ_SIZE
This is the maximum amount we'll read from any one file before moving on to the next. This way, we get quick response time for other files if one file is growing rapidly.

See Also:
Constant Field Values

MAX_READ_SIZE_OPT

public static final String MAX_READ_SIZE_OPT
See Also:
Constant Field Values

MAX_READ_SIZE

public static int MAX_READ_SIZE

tailer

protected static org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.FileTailer tailer

fileReadOffset

protected long fileReadOffset
next PHYSICAL offset to read


offsetOfFirstByte

protected long offsetOfFirstByte
The logical offset of the first byte of the file


conf

protected org.apache.hadoop.conf.Configuration conf
Constructor Detail

LWFTAdaptor

public LWFTAdaptor()
Method Detail

start

public void start(long offset)
Specified by:
start in class AbstractAdaptor

getCurrentStatus

public String getCurrentStatus()
Description copied from interface: Adaptor
Return the adaptor's state Should not include class name or byte offset, which are written by caller. The datatype should, however, be written by this method.

Returns:
the adaptor state as a string
See Also:
Adaptor.getCurrentStatus()

toString

public String toString()
Overrides:
toString in class Object

getStreamName

public String getStreamName()

parseArgs

public String parseArgs(String params)
Description copied from interface: Adaptor
Parse args, return stream name. Do not start running. Return the stream name, given params. The stream name is the part of the Adaptor status that's used to determine uniqueness.

Returns:
Stream name as a string, null if params are malformed

hardStop

public void hardStop()
              throws AdaptorException
Description copied from interface: Adaptor
Signals this adaptor to come to an abrupt stop, as quickly as it can. The use case here is "Whups, I didn't mean to start that adaptor tailing a gigabyte file, stop it now". Adaptors might need to do something nontrivial here, e.g., in the case in which they have registered periodic timer interrupts, or use a shared worker thread from which they need to disengage. This method is synchronous: In other words, after shutdown() returns, no new data should be written.

Throws:
AdaptorException

shutdown

public long shutdown()
              throws AdaptorException
Description copied from interface: Adaptor
Signals this adaptor to come to an orderly stop. The adaptor ought to push out all the data it can before exiting. This method is synchronous up to 60 seconds

Returns:
the logical offset at which the adaptor stops
Throws:
AdaptorException

shutdown

public long shutdown(AdaptorShutdownPolicy shutdownPolicy)
              throws AdaptorException
Description copied from interface: Adaptor
Signals this adaptor to come to an orderly stop. The adaptor ought to push out all the data it can before exiting depending of the shutdown policy

Returns:
the logical offset at which the adaptor was when the method return
Throws:
AdaptorException

extractRecords

protected int extractRecords(ChunkReceiver eq,
                             long buffOffsetInFile,
                             byte[] buf)
                      throws InterruptedException
Extract records from a byte sequence

Parameters:
eq - the queue to stick the new chunk[s] in
buffOffsetInFile - the byte offset in the stream at which buf[] begins
buf - the byte buffer to extract records from
Returns:
the number of bytes processed
Throws:
InterruptedException

slurp

protected boolean slurp(long len,
                        RandomAccessFile reader)
                 throws IOException,
                        InterruptedException
Throws:
IOException
InterruptedException

tailFile

public boolean tailFile(ChunkReceiver eq)
                 throws InterruptedException
Throws:
InterruptedException


Copyright © ${year} The Apache Software Foundation