kafka.log

LogManager

class LogManager extends Logging

The entry point to the kafka log management subsystem. The log manager is responsible for log creation, retrieval, and cleaning. All read and write operations are delegated to the individual log instances.

The log manager maintains logs in one or more directories. New logs are created in the data directory with the fewest logs. No attempt is made to move partitions after the fact or balance based on size or I/O rate.

A background thread handles log retention by periodically truncating excess log segments.

Annotations
@threadsafe()
Linear Supertypes
Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. LogManager
  2. Logging
  3. AnyRef
  4. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new LogManager(logDirs: Array[File], topicConfigs: Map[String, LogConfig], defaultConfig: LogConfig, cleanerConfig: CleanerConfig, ioThreads: Int, flushCheckMs: Long, flushCheckpointMs: Long, retentionCheckMs: Long, scheduler: Scheduler, brokerState: BrokerState, time: Time)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. val InitialTaskDelayMs: Int

  7. val LockFile: String

  8. val RecoveryPointCheckpointFile: String

  9. def allLogs(): Iterable[Log]

    Get all the partition logs

  10. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  11. val brokerState: BrokerState

  12. def checkpointRecoveryPointOffsets(): Unit

    Write out the current recovery point for all logs to a text file in the log directory to avoid recovering the whole log on startup.

  13. val cleaner: LogCleaner

  14. val cleanerConfig: CleanerConfig

  15. def cleanupLogs(): Unit

    Delete any eligible logs.

    Delete any eligible logs. Return the number of segments deleted.

  16. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  17. def createLog(topicAndPartition: TopicAndPartition, config: LogConfig): Log

    Create a log for the given topic and the given partition If the log already exists, just return a copy of the existing log

  18. def debug(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  19. def debug(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  20. def debug(msg: ⇒ String): Unit

    Definition Classes
    Logging
  21. val defaultConfig: LogConfig

  22. def deleteLog(topicAndPartition: TopicAndPartition): Unit

    Delete a log.

  23. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  24. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  25. def error(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  26. def error(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  27. def error(msg: ⇒ String): Unit

    Definition Classes
    Logging
  28. def fatal(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  29. def fatal(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  30. def fatal(msg: ⇒ String): Unit

    Definition Classes
    Logging
  31. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  32. val flushCheckMs: Long

  33. val flushCheckpointMs: Long

  34. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  35. def getLog(topicAndPartition: TopicAndPartition): Option[Log]

    Get the log if it exists, otherwise return None

  36. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  37. def info(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  38. def info(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  39. def info(msg: ⇒ String): Unit

    Definition Classes
    Logging
  40. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  41. val logDirs: Array[File]

  42. var logIdent: String

    Attributes
    protected
    Definition Classes
    Logging
  43. lazy val logger: Logger

    Definition Classes
    Logging
  44. val loggerName: String

    Definition Classes
    Logging
  45. def logsByTopicPartition: Map[TopicAndPartition, Log]

    Get a map of TopicAndPartition => Log

  46. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  47. final def notify(): Unit

    Definition Classes
    AnyRef
  48. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  49. val retentionCheckMs: Long

  50. def shutdown(): Unit

    Close all the logs

  51. def startup(): Unit

    Start the background threads to flush logs and do log cleanup

  52. def swallow(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  53. def swallowDebug(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  54. def swallowError(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  55. def swallowInfo(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  56. def swallowTrace(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  57. def swallowWarn(action: ⇒ Unit): Unit

    Definition Classes
    Logging
  58. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  59. def toString(): String

    Definition Classes
    AnyRef → Any
  60. val topicConfigs: Map[String, LogConfig]

  61. def trace(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  62. def trace(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  63. def trace(msg: ⇒ String): Unit

    Definition Classes
    Logging
  64. def truncateFullyAndStartAt(topicAndPartition: TopicAndPartition, newOffset: Long): Unit

    Delete all data in a partition and start the log at the new offset

    Delete all data in a partition and start the log at the new offset

    newOffset

    The new offset to start the log with

  65. def truncateTo(partitionAndOffsets: Map[TopicAndPartition, Long]): Unit

    Truncate the partition logs to the specified offsets and checkpoint the recovery point to this offset

    Truncate the partition logs to the specified offsets and checkpoint the recovery point to this offset

    partitionAndOffsets

    Partition logs that need to be truncated

  66. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  67. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  68. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  69. def warn(msg: ⇒ String, e: ⇒ Throwable): Unit

    Definition Classes
    Logging
  70. def warn(e: ⇒ Throwable): Any

    Definition Classes
    Logging
  71. def warn(msg: ⇒ String): Unit

    Definition Classes
    Logging

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped