Log

Instance Constructors

new Log(dir: File, config: LogConfig, recoveryPoint: Long = 0L, scheduler: Scheduler, time: Time = SystemTime)

dir
The directory in which log segments are created.
config
The log configuration settings
recoveryPoint
The offset at which to begin recovery--i.e. the first offset which has not been flushed to disk
scheduler
The thread pool scheduler used for background actions
time
The time instance used for checking the clock

Type Members

case class LogAppendInfo(firstOffset: Long, lastOffset: Long, codec: CompressionCodec, shallowCount: Int, validBytes: Int, offsetsMonotonic: Boolean) extends Product with Serializable

Struct to hold various quantities we compute about each message set before appending to the log
Struct to hold various quantities we compute about each message set before appending to the log
firstOffset
The first offset in the message set
lastOffset
The last offset in the message set
codec
The codec used in the message set
shallowCount
The number of shallow messages
validBytes
The number of valid bytes
offsetsMonotonic
Are the offsets in this message set monotonically increasing

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def activeSegment: LogSegment

The active segment that is currently taking appends
def addSegment(segment: LogSegment): LogSegment

Add the given segment to the segments in this log.
Add the given segment to the segments in this log. If this segment replaces an existing segment, delete it.
segment
The segment to add
def append(messages: ByteBufferMessageSet, assignOffsets: Boolean = true): LogAppendInfo

Append this message set to the active segment of the log, rolling over to a fresh segment if necessary.
Append this message set to the active segment of the log, rolling over to a fresh segment if necessary.
This method will generally be responsible for assigning offsets to the messages, however if the assignOffsets=false flag is passed we will only check that the existing offsets are valid.
messages
The message set to append
assignOffsets
Should the log assign offsets to this message set or blindly apply what it is given
returns
Information about the appended messages including the first and last offset.

Exceptions thrown
KafkaStorageException
If the append fails due to an I/O error.
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
def close(): Unit

Close this log
var config: LogConfig

The log configuration settings
def convertToOffsetMetadata(offset: Long): LogOffsetMetadata

Given a message offset, find its corresponding offset metadata in the log.
Given a message offset, find its corresponding offset metadata in the log. If the message offset is out of range, return unknown offset metadata
def debug(msg: ⇒ String, e: ⇒ Throwable): Unit

Definition Classes
Logging
def debug(e: ⇒ Throwable): Any

Definition Classes
Logging
def debug(msg: ⇒ String): Unit

Definition Classes
Logging
def deleteOldSegments(predicate: (LogSegment) ⇒ Boolean): Int

Delete any log segments matching the given predicate function, starting with the oldest segment and moving forward until a segment doesn't match.
Delete any log segments matching the given predicate function, starting with the oldest segment and moving forward until a segment doesn't match.
predicate
A function that takes in a single log segment and returns true iff it is deletable
returns
The number of segments deleted
val dir: File

The directory in which log segments are created.
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def error(msg: ⇒ String, e: ⇒ Throwable): Unit

Definition Classes
Logging
def error(e: ⇒ Throwable): Any

Definition Classes
Logging
def error(msg: ⇒ String): Unit

Definition Classes
Logging
def fatal(msg: ⇒ String, e: ⇒ Throwable): Unit

Definition Classes
Logging
def fatal(e: ⇒ Throwable): Any

Definition Classes
Logging
def fatal(msg: ⇒ String): Unit

Definition Classes
Logging
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
def flush(offset: Long): Unit

Flush log segments for all offsets up to offset-1
Flush log segments for all offsets up to offset-1
offset
The offset to flush up to (non-inclusive); the new recovery point
def flush(): Unit

Flush all log segments
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
def info(msg: ⇒ String, e: ⇒ Throwable): Unit

Definition Classes
Logging
def info(e: ⇒ Throwable): Any

Definition Classes
Logging
def info(msg: ⇒ String): Unit

Definition Classes
Logging
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def lastFlushTime(): Long

The time this log is last known to have been fully flushed to disk
def logEndOffset: Long

The offset of the next message that will be appended to the log
def logEndOffsetMetadata: LogOffsetMetadata

The offset metadata of the next message that will be appended to the log
var logIdent: String

Attributes
protected
Definition Classes
Logging
def logSegments(from: Long, to: Long): Iterable[LogSegment]

Get all segments beginning with the segment that includes "from" and ending with the segment that includes up to "to-1" or the end of the log (if to > logEndOffset)
def logSegments: Iterable[LogSegment]

All the log segments in this log ordered from oldest to newest
def logStartOffset: Long

The earliest message offset in the log
lazy val logger: Logger

Definition Classes
Logging
val loggerName: String

Definition Classes
Logging
def name: String

The name of this log
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def newGauge[T](name: String, metric: Gauge[T]): Gauge[T]

Definition Classes
KafkaMetricsGroup
def newHistogram(name: String, biased: Boolean = true): Histogram

Definition Classes
KafkaMetricsGroup
def newMeter(name: String, eventType: String, timeUnit: TimeUnit): Meter

Definition Classes
KafkaMetricsGroup
def newTimer(name: String, durationUnit: TimeUnit, rateUnit: TimeUnit): Timer

Definition Classes
KafkaMetricsGroup
var nextOffsetMetadata: LogOffsetMetadata
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def numberOfSegments: Int

The number of segments in the log.
The number of segments in the log. Take care! this is an O(n) operation.
def read(startOffset: Long, maxLength: Int, maxOffset: Option[Long] = None): FetchDataInfo

Read messages from the log
Read messages from the log
startOffset
The offset to begin reading at
maxLength
The maximum number of bytes to read
maxOffset
-The offset to read up to, exclusive. (i.e. the first offset NOT included in the resulting message set).
returns
The fetch data information including fetch starting offset metadata and messages read

Exceptions thrown
OffsetOutOfRangeException
If startOffset is beyond the log end offset or before the base offset of the first segment.
var recoveryPoint: Long

The offset at which to begin recovery--i.
The offset at which to begin recovery--i.e. the first offset which has not been flushed to disk
def removeMetric(name: String): Unit

Definition Classes
KafkaMetricsGroup
def roll(): LogSegment

Roll the log over to a new active segment starting with the current logEndOffset.
Roll the log over to a new active segment starting with the current logEndOffset. This will trim the index to the exact size of the number of entries it currently contains.
returns
The newly rolled segment
def size: Long

The size of the log in bytes
def swallow(action: ⇒ Unit): Unit

Definition Classes
Logging
def swallowDebug(action: ⇒ Unit): Unit

Definition Classes
Logging
def swallowError(action: ⇒ Unit): Unit

Definition Classes
Logging
def swallowInfo(action: ⇒ Unit): Unit

Definition Classes
Logging
def swallowTrace(action: ⇒ Unit): Unit

Definition Classes
Logging
def swallowWarn(action: ⇒ Unit): Unit

Definition Classes
Logging
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
Log → AnyRef → Any
val topicAndPartition: TopicAndPartition
def trace(msg: ⇒ String, e: ⇒ Throwable): Unit

Definition Classes
Logging
def trace(e: ⇒ Throwable): Any

Definition Classes
Logging
def trace(msg: ⇒ String): Unit

Definition Classes
Logging
def unflushedMessages(): Long

The number of messages appended to the log since the last flush
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
def warn(msg: ⇒ String, e: ⇒ Throwable): Unit

Definition Classes
Logging
def warn(e: ⇒ Throwable): Any

Definition Classes
Logging
def warn(msg: ⇒ String): Unit

Definition Classes
Logging

class Log extends Logging with KafkaMetricsGroup

Instance Constructors

new Log(dir: File, config: LogConfig, recoveryPoint: Long = 0L, scheduler: Scheduler, time: Time = SystemTime)

Type Members

case class LogAppendInfo(firstOffset: Long, lastOffset: Long, codec: CompressionCodec, shallowCount: Int, validBytes: Int, offsetsMonotonic: Boolean) extends Product with Serializable

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

def activeSegment: LogSegment

def addSegment(segment: LogSegment): LogSegment

def append(messages: ByteBufferMessageSet, assignOffsets: Boolean = true): LogAppendInfo

final def asInstanceOf[T0]: T0

def clone(): AnyRef

def close(): Unit

var config: LogConfig

def convertToOffsetMetadata(offset: Long): LogOffsetMetadata

def debug(msg: ⇒ String, e: ⇒ Throwable): Unit

def debug(e: ⇒ Throwable): Any

def debug(msg: ⇒ String): Unit

def deleteOldSegments(predicate: (LogSegment) ⇒ Boolean): Int

val dir: File

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def error(msg: ⇒ String, e: ⇒ Throwable): Unit

def error(e: ⇒ Throwable): Any

def error(msg: ⇒ String): Unit

def fatal(msg: ⇒ String, e: ⇒ Throwable): Unit

def fatal(e: ⇒ Throwable): Any

def fatal(msg: ⇒ String): Unit

def finalize(): Unit

def flush(offset: Long): Unit

def flush(): Unit

final def getClass(): Class[_]

def hashCode(): Int

def info(msg: ⇒ String, e: ⇒ Throwable): Unit

def info(e: ⇒ Throwable): Any

def info(msg: ⇒ String): Unit

final def isInstanceOf[T0]: Boolean

def lastFlushTime(): Long

def logEndOffset: Long

def logEndOffsetMetadata: LogOffsetMetadata

var logIdent: String

def logSegments(from: Long, to: Long): Iterable[LogSegment]

def logSegments: Iterable[LogSegment]

def logStartOffset: Long

lazy val logger: Logger

val loggerName: String

def name: String

final def ne(arg0: AnyRef): Boolean

def newGauge[T](name: String, metric: Gauge[T]): Gauge[T]

def newHistogram(name: String, biased: Boolean = true): Histogram

def newMeter(name: String, eventType: String, timeUnit: TimeUnit): Meter

def newTimer(name: String, durationUnit: TimeUnit, rateUnit: TimeUnit): Timer

var nextOffsetMetadata: LogOffsetMetadata

final def notify(): Unit

final def notifyAll(): Unit

def numberOfSegments: Int

def read(startOffset: Long, maxLength: Int, maxOffset: Option[Long] = None): FetchDataInfo

var recoveryPoint: Long

def removeMetric(name: String): Unit

def roll(): LogSegment

def size: Long

def swallow(action: ⇒ Unit): Unit

def swallowDebug(action: ⇒ Unit): Unit

def swallowError(action: ⇒ Unit): Unit

def swallowInfo(action: ⇒ Unit): Unit

def swallowTrace(action: ⇒ Unit): Unit

def swallowWarn(action: ⇒ Unit): Unit

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

val topicAndPartition: TopicAndPartition

def trace(msg: ⇒ String, e: ⇒ Throwable): Unit

def trace(e: ⇒ Throwable): Any

def trace(msg: ⇒ String): Unit

def unflushedMessages(): Long

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit