consumer

Type Members

1. class AssignmentContext extends AnyRef

   

2. trait BaseConsumer extends AnyRef

   

   A base consumer used to abstract both old and new consumer this class should be removed (along with BaseProducer) be removed once we deprecate old consumer

3. case class BaseConsumerRecord(topic: String, partition: Int, offset: Long, key: Array[Byte], value: Array[Byte]) extends Product with Serializable

   

4. case class Blacklist(rawRegex: String) extends TopicFilter with Product with Serializable

   

5. class ConsumerConfig extends ZKConfig

   

6. trait ConsumerConnector extends AnyRef

   

   Main interface for consumer

7. class ConsumerFetcherManager extends AbstractFetcherManager

   

   Usage: Once ConsumerFetcherManager is created, startConnections() and stopAllConnections() can be called repeatedly until shutdown() is called.

8. class ConsumerFetcherThread extends AbstractFetcherThread

   

9. class ConsumerIterator[K, V] extends IteratorTemplate[MessageAndMetadata[K, V]] with Logging

   

   An iterator that blocks until a value can be read from the supplied queue.

   An iterator that blocks until a value can be read from the supplied queue. The iterator takes a shutdownCommand object which can be added to the queue to trigger a shutdown

10. case class ConsumerThreadId(consumer: String, threadId: Int) extends Ordered[ConsumerThreadId] with Product with Serializable

    

11. class ConsumerTimeoutException extends RuntimeException

    

12. class ConsumerTopicMetrics extends KafkaMetricsGroup

    

    Annotations

    @threadsafe()

13. class ConsumerTopicStats extends Logging

    

    Tracks metrics for each topic the given consumer client has consumed data from.

14. class FetchRequestAndResponseMetrics extends KafkaMetricsGroup

    

15. class FetchRequestAndResponseStats extends AnyRef

    

    Tracks metrics of the requests made by a given consumer client to all brokers, and the responses obtained from the brokers.

16. case class FetchedDataChunk(messages: ByteBufferMessageSet, topicInfo: PartitionTopicInfo, fetchOffset: Long) extends Product with Serializable

    

17. class KafkaStream[K, V] extends Iterable[MessageAndMetadata[K, V]] with Iterable[MessageAndMetadata[K, V]]

    

18. class NewShinyConsumer extends BaseConsumer

    

19. class OldConsumer extends BaseConsumer

    

20. trait PartitionAssignor extends AnyRef

    

21. class PartitionTopicInfo extends Logging

    

22. class RangeAssignor extends PartitionAssignor with Logging

    

    Range partitioning works on a per-topic basis.

    Range partitioning works on a per-topic basis. For each topic, we lay out the available partitions in numeric order and the consumer threads in lexicographic order. We then divide the number of partitions by the total number of consumer streams (threads) to determine the number of partitions to assign to each consumer. If it does not evenly divide, then the first few consumers will have one extra partition. For example, suppose there are two consumers C1 and C2 with two streams each, and there are five available partitions (p0, p1, p2, p3, p4). So each consumer thread will get at least one partition and the first consumer thread will get one extra partition. So the assignment will be: p0 -> C1-0, p1 -> C1-0, p2 -> C1-1, p3 -> C2-0, p4 -> C2-1

23. class RoundRobinAssignor extends PartitionAssignor with Logging

    

    The round-robin partition assignor lays out all the available partitions and all the available consumer threads.

    The round-robin partition assignor lays out all the available partitions and all the available consumer threads. It then proceeds to do a round-robin assignment from partition to consumer thread. If the subscriptions of all consumer instances are identical, then the partitions will be uniformly distributed. (i.e., the partition ownership counts will be within a delta of exactly one across all consumer threads.)

    (For simplicity of implementation) the assignor is allowed to assign a given topic-partition to any consumer instance and thread-id within that instance. Therefore, round-robin assignment is allowed only if: a) Every topic has the same number of streams within a consumer instance b) The set of subscribed topics is identical for every consumer instance within the group.

24. class SimpleConsumer extends Logging

    

    A consumer of kafka messages

    A consumer of kafka messages

    Annotations

    @threadsafe()

25. trait TopicEventHandler[T] extends AnyRef

    

26. sealed abstract class TopicFilter extends Logging

    

27. case class Whitelist(rawRegex: String) extends TopicFilter with Product with Serializable

    

28. class ZookeeperTopicEventWatcher extends Logging