kafka.log

Log

class Log extends Logging with KafkaMetricsGroup

An append-only log for storing messages.

The log is a sequence of LogSegments, each with a base offset denoting the first message in the segment.

New log segments are created according to a configurable policy that controls the size in bytes or time interval for a given segment.

Annotations
@threadsafe()
Linear Supertypes
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Hide All
  2. Show all
  1. Log
  2. KafkaMetricsGroup
  3. Logging
  4. AnyRef
  5. Any
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Log(dir: File, config: LogConfig, recoveryPoint: Long = 0L, scheduler: Scheduler, time: Time = kafka.utils.SystemTime)

    dir

    The directory in which log segments are created.

    config

    The log configuration settings

    recoveryPoint

    The offset at which to begin recovery--i.e. the first offset which has not been flushed to disk

    scheduler

    The thread pool scheduler used for background actions

    time

    The time instance used for checking the clock

Type Members

  1. case class LogAppendInfo(firstOffset: Long, lastOffset: Long, codec: CompressionCodec, shallowCount: Int, offsetsMonotonic: Boolean) extends Product with Serializable

    Struct to hold various quantities we compute about each message set before appending to the log

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def activeSegment: LogSegment

    The active segment that is currently taking appends

  7. def addSegment(segment: LogSegment): LogSegment

    Add the given segment to the segments in this log.

    Add the given segment to the segments in this log. If this segment replaces an existing segment, delete it.

    segment

    The segment to add

  8. def append(messages: ByteBufferMessageSet, assignOffsets: Boolean = true): LogAppendInfo

    Append this message set to the active segment of the log, rolling over to a fresh segment if necessary.

    Append this message set to the active segment of the log, rolling over to a fresh segment if necessary.

    This method will generally be responsible for assigning offsets to the messages, however if the assignOffsets=false flag is passed we will only check that the existing offsets are valid.

    messages

    The message set to append

    assignOffsets

    Should the log assign offsets to this message set or blindly apply what it is given

    returns

    Information about the appended messages including the first and last offset.

    Exceptions thrown
    KafkaStorageException

    If the append fails due to an I/O error.

  9. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  10. def clone(): AnyRef

    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  11. def close(): Unit

    Close this log

  12. var config: LogConfig

    The log configuration settings

  13. def debug(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  14. def debug(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  15. def debug(msg: ⇒ String): Unit

    Definition Classes
    Logging
  16. def deleteOldSegments(predicate: (LogSegment) ⇒ Boolean): Int

    Delete any log segments matching the given predicate function, starting with the oldest segment and moving forward until a segment doesn't match.

    Delete any log segments matching the given predicate function, starting with the oldest segment and moving forward until a segment doesn't match.

    predicate

    A function that takes in a single log segment and returns true iff it is deletable

    returns

    The number of segments deleted

  17. val dir: File

    The directory in which log segments are created.

  18. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  19. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  20. def error(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  21. def error(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  22. def error(msg: ⇒ String): Unit

    Definition Classes
    Logging
  23. def fatal(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  24. def fatal(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  25. def fatal(msg: ⇒ String): Unit

    Definition Classes
    Logging
  26. def finalize(): Unit

    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  27. def flush(offset: Long): Unit

    Flush log segments for all offsets up to offset-1

    Flush log segments for all offsets up to offset-1

    offset

    The offset to flush up to (non-inclusive); the new recovery point

  28. def flush(): Unit

    Flush all log segments

  29. final def getClass(): java.lang.Class[_]

    Definition Classes
    AnyRef → Any
  30. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  31. def info(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  32. def info(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  33. def info(msg: ⇒ String): Unit

    Definition Classes
    Logging
  34. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  35. def lastFlushTime(): Long

    The time this log is last known to have been fully flushed to disk

  36. def logEndOffset: Long

    The offset of the next message that will be appended to the log

  37. var logIdent: String

    Attributes
    protected
    Definition Classes
    Logging
  38. def logSegments(from: Long, to: Long): Iterable[LogSegment]

    Get all segments beginning with the segment that includes "from" and ending with the segment that includes up to "to-1" or the end of the log (if to > logEndOffset)

  39. def logSegments: Iterable[LogSegment]

    All the log segments in this log ordered from oldest to newest

  40. lazy val logger: Logger

    Definition Classes
    Logging
  41. val loggerName: String

    Definition Classes
    Logging
  42. def name: String

    The name of this log

  43. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  44. def newGauge[T](name: String, metric: Gauge[T]): Gauge[T]

    Definition Classes
    KafkaMetricsGroup
  45. def newHistogram(name: String, biased: Boolean = true): Histogram

    Definition Classes
    KafkaMetricsGroup
  46. def newMeter(name: String, eventType: String, timeUnit: TimeUnit): Meter

    Definition Classes
    KafkaMetricsGroup
  47. def newTimer(name: String, durationUnit: TimeUnit, rateUnit: TimeUnit): Timer

    Definition Classes
    KafkaMetricsGroup
  48. final def notify(): Unit

    Definition Classes
    AnyRef
  49. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  50. def numberOfSegments: Int

    The number of segments in the log.

    The number of segments in the log. Take care! this is an O(n) operation.

  51. def read(startOffset: Long, maxLength: Int, maxOffset: Option[Long] = None): MessageSet

    Read messages from the log

    Read messages from the log

    startOffset

    The offset to begin reading at

    maxLength

    The maximum number of bytes to read

    maxOffset

    -The offset to read up to, exclusive. (i.e. the first offset NOT included in the resulting message set).

    returns

    The messages read

    Exceptions thrown
    OffsetOutOfRangeException

    If startOffset is beyond the log end offset or before the base offset of the first segment.

  52. var recoveryPoint: Long

    The offset at which to begin recovery--i.

    The offset at which to begin recovery--i.e. the first offset which has not been flushed to disk

  53. def roll(): LogSegment

    Roll the log over to a new active segment starting with the current logEndOffset.

    Roll the log over to a new active segment starting with the current logEndOffset. This will trim the index to the exact size of the number of entries it currently contains.

    returns

    The newly rolled segment

  54. val scheduler: Scheduler

    The thread pool scheduler used for background actions

  55. def size: Long

    The size of the log in bytes

  56. def swallow(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  57. def swallowDebug(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  58. def swallowError(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  59. def swallowInfo(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  60. def swallowTrace(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  61. def swallowWarn(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  62. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  63. def toString(): String

    Definition Classes
    Log → AnyRef → Any
  64. val topicAndPartition: TopicAndPartition

  65. def trace(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  66. def trace(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  67. def trace(msg: ⇒ String): Unit

    Definition Classes
    Logging
  68. def unflushedMessages(): Long

    The number of messages appended to the log since the last flush

  69. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  70. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  71. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  72. def warn(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  73. def warn(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  74. def warn(msg: ⇒ String): Unit

    Definition Classes
    Logging

Inherited from KafkaMetricsGroup

Inherited from Logging

Inherited from AnyRef

Inherited from Any