kafka

consumer

package consumer

Visibility
  1. Public
  2. All

Type Members

  1. class AssignmentContext extends AnyRef

  2. case class Blacklist(rawRegex: String) extends TopicFilter with Product with Serializable

  3. class ConsumerConfig extends ZKConfig

  4. trait ConsumerConnector extends AnyRef

    Main interface for consumer

  5. class ConsumerFetcherManager extends AbstractFetcherManager

    Usage: Once ConsumerFetcherManager is created, startConnections() and stopAllConnections() can be called repeatedly until shutdown() is called.

  6. class ConsumerFetcherThread extends AbstractFetcherThread

  7. class ConsumerIterator[K, V] extends IteratorTemplate[MessageAndMetadata[K, V]] with Logging

    An iterator that blocks until a value can be read from the supplied queue.

    An iterator that blocks until a value can be read from the supplied queue. The iterator takes a shutdownCommand object which can be added to the queue to trigger a shutdown

  8. case class ConsumerThreadId(consumer: String, threadId: Int) extends Ordered[ConsumerThreadId] with Product with Serializable

  9. class ConsumerTimeoutException extends RuntimeException

  10. class ConsumerTopicMetrics extends KafkaMetricsGroup

    Annotations
    @threadsafe()
  11. class ConsumerTopicStats extends Logging

    Tracks metrics for each topic the given consumer client has consumed data from.

  12. class FetchRequestAndResponseMetrics extends KafkaMetricsGroup

  13. class FetchRequestAndResponseStats extends AnyRef

    Tracks metrics of the requests made by a given consumer client to all brokers, and the responses obtained from the brokers.

  14. case class FetchedDataChunk(messages: ByteBufferMessageSet, topicInfo: PartitionTopicInfo, fetchOffset: Long) extends Product with Serializable

  15. class KafkaStream[K, V] extends Iterable[MessageAndMetadata[K, V]] with Iterable[MessageAndMetadata[K, V]]

  16. trait PartitionAssignor extends AnyRef

  17. class PartitionTopicInfo extends Logging

  18. class RangeAssignor extends PartitionAssignor with Logging

    Range partitioning works on a per-topic basis.

    Range partitioning works on a per-topic basis. For each topic, we lay out the available partitions in numeric order and the consumer threads in lexicographic order. We then divide the number of partitions by the total number of consumer streams (threads) to determine the number of partitions to assign to each consumer. If it does not evenly divide, then the first few consumers will have one extra partition. For example, suppose there are two consumers C1 and C2 with two streams each, and there are five available partitions (p0, p1, p2, p3, p4). So each consumer thread will get at least one partition and the first consumer thread will get one extra partition. So the assignment will be: p0 -> C1-0, p1 -> C1-0, p2 -> C1-1, p3 -> C2-0, p4 -> C2-1

  19. class RoundRobinAssignor extends PartitionAssignor with Logging

    The round-robin partition assignor lays out all the available partitions and all the available consumer threads.

    The round-robin partition assignor lays out all the available partitions and all the available consumer threads. It then proceeds to do a round-robin assignment from partition to consumer thread. If the subscriptions of all consumer instances are identical, then the partitions will be uniformly distributed. (i.e., the partition ownership counts will be within a delta of exactly one across all consumer threads.)

    (For simplicity of implementation) the assignor is allowed to assign a given topic-partition to any consumer instance and thread-id within that instance. Therefore, round-robin assignment is allowed only if: a) Every topic has the same number of streams within a consumer instance b) The set of subscribed topics is identical for every consumer instance within the group.

  20. class SimpleConsumer extends Logging

    A consumer of kafka messages

    A consumer of kafka messages

    Annotations
    @threadsafe()
  21. trait TopicEventHandler[T] extends AnyRef

  22. sealed abstract class TopicFilter extends Logging

  23. case class Whitelist(rawRegex: String) extends TopicFilter with Product with Serializable

  24. class ZookeeperTopicEventWatcher extends Logging

Value Members

  1. object Consumer extends Logging

  2. object ConsumerConfig extends Config

  3. object ConsumerTopicStatsRegistry

    Stores the topic stats information of each consumer client in a (clientId -> ConsumerTopicStats) map.

  4. object FetchRequestAndResponseStatsRegistry

    Stores the fetch request and response stats information of each consumer client in a (clientId -> FetchRequestAndResponseStats) map.

  5. object PartitionAssignor

  6. object PartitionTopicInfo

Ungrouped