LogManager
Related Doc:
package log
Get all the partition logs
Write out the current recovery point for all logs to a text file in the log directory to avoid recovering the whole log on startup.
Delete any eligible logs.
Delete any eligible logs. Return the number of segments deleted.
Create a log for the given topic and the given partition If the log already exists, just return a copy of the existing log
Delete a log.
Get the log if it exists, otherwise return None
Get a map of TopicAndPartition => Log
Close all the logs
Start the background threads to flush logs and do log cleanup
Delete all data in a partition and start the log at the new offset
Delete all data in a partition and start the log at the new offset
The new offset to start the log with
Truncate the partition logs to the specified offsets and checkpoint the recovery point to this offset
Truncate the partition logs to the specified offsets and checkpoint the recovery point to this offset
Partition logs that need to be truncated
The entry point to the kafka log management subsystem. The log manager is responsible for log creation, retrieval, and cleaning. All read and write operations are delegated to the individual log instances.
The log manager maintains logs in one or more directories. New logs are created in the data directory with the fewest logs. No attempt is made to move partitions after the fact or balance based on size or I/O rate.
A background thread handles log retention by periodically truncating excess log segments.