Release Notes - Kafka - Version 0.10.0.0

Sub-task

[KAFKA-2068] - Replace OffsetCommit Request/Response with org.apache.kafka.common.requests equivalent
[KAFKA-2069] - Replace OffsetFetch request/response with their org.apache.kafka.common.requests equivalent
[KAFKA-2070] - Replace OffsetRequest/response with ListOffsetRequest/response from org.apache.kafka.common.requests
[KAFKA-2071] - Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents
[KAFKA-2072] - Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module
[KAFKA-2073] - Replace TopicMetadata request/response with o.a.k.requests.metadata
[KAFKA-2370] - Add pause/unpause connector support
[KAFKA-2422] - Allow copycat connector plugins to be aliased to simpler names
[KAFKA-2479] - Add CopycatExceptions to indicate transient and permanent errors in a connector/task
[KAFKA-2508] - Replace UpdateMetadata{Request,Response} with org.apache.kafka.common.requests equivalent
[KAFKA-2509] - Replace LeaderAndIsr{Request,Response} with org.apache.kafka.common.network.requests equivalent
[KAFKA-2591] - Remove Persistent Data before Restoringafter a Fault
[KAFKA-2592] - Stop Writing the Change-log in store.put() / delete() for Non-transactional Store
[KAFKA-2593] - KeyValueStores should not require use of the context's default serializers and deserializers
[KAFKA-2594] - Add a key-value store that is a fixed-capacity in-memory LRU cache
[KAFKA-2600] - Make KStream interfaces compatible with Java 8 java.util.function
[KAFKA-2642] - Run replication tests in ducktape with SSL for clients
[KAFKA-2643] - Run mirror maker tests in ducktape with SSL and SASL
[KAFKA-2649] - Add support for custom partitioner in sink nodes
[KAFKA-2652] - Incorporate the new consumer protocol with partition-group interface
[KAFKA-2653] - Stateful operations in the KStream DSL layer
[KAFKA-2654] - Avoid calling Consumer.poll(0) in each iteration
[KAFKA-2658] - Implement SASL/PLAIN
[KAFKA-2667] - Copycat KafkaBasedLogTest.testSendAndReadToEnd transient failure
[KAFKA-2693] - Run relevant ducktape tests with SASL/PLAIN and multiple mechanisms
[KAFKA-2694] - Make a task id be a composite id of a topic group id and a partition id
[KAFKA-2698] - add paused API
[KAFKA-2706] - Make state stores first class citizens in the processor DAG
[KAFKA-2707] - Make KStream processor names deterministic
[KAFKA-2718] - Reuse of temporary directories leading to transient unit test failures
[KAFKA-2727] - initialize only the part of the topology relevant to the task
[KAFKA-2733] - Distinguish metric names inside the sensor registry
[KAFKA-2763] - Reduce stream task migrations and initialization costs
[KAFKA-2802] - Add integration tests for Kafka Streams
[KAFKA-2804] - Create / Update changelog topics upon state store initialization
[KAFKA-2811] - Add standby tasks
[KAFKA-2837] - FAILING TEST: kafka.api.ProducerBounceTest > testBrokerFailure
[KAFKA-2839] - Kafka connect log test failing
[KAFKA-2856] - add KTable
[KAFKA-2910] - Failure in kafka.api.SslEndToEndAuthorizationTest.testNoGroupAcl
[KAFKA-2962] - Add Simple Join API
[KAFKA-2982] - Mark the old Scala producer and related classes as deprecated
[KAFKA-2984] - KTable should send old values along with new values to downstreams
[KAFKA-3016] - Add KStream-KStream window joins
[KAFKA-3020] - Ensure Checkstyle runs on all Java code
[KAFKA-3021] - Centralize dependency version managment
[KAFKA-3022] - Deduplicate common project configurations
[KAFKA-3025] - KIP-31&KIP-32 (part 1): Add timestamp field to message, configs, and Producer/ConsumerRecord
[KAFKA-3026] - KIP-32 (part 2): Changes in broker to over-write timestamp or reject message
[KAFKA-3030] - Remove unused scala dependencies
[KAFKA-3036] - Add up-conversion and down-conversion of ProducerRequest and FetchRequest to broker.
[KAFKA-3060] - Refactor MeteredXXStore
[KAFKA-3063] - LogRecoveryTest exits with -1 occasionally
[KAFKA-3066] - Add Demo Examples for Kafka Streams
[KAFKA-3078] - Add ducktape tests for KafkaLog4jAppender producing to SASL enabled Kafka cluster
[KAFKA-3081] - KTable Aggregation Implementation
[KAFKA-3104] - Windowed Stream Aggregation Implementation
[KAFKA-3108] - KStream custom StreamPartitioner for windowed key
[KAFKA-3121] - KStream DSL API Improvement
[KAFKA-3125] - Exception Hierarchy for Streams
[KAFKA-3133] - Add putIfAbsent function to KeyValueStore
[KAFKA-3136] - Rename KafkaStreaming to KafkaStreams
[KAFKA-3142] - Improve error message in kstreams
[KAFKA-3153] - Serializer/Deserializer Registration and Type inference
[KAFKA-3165] - Fix ignored parameters in the gradle "All" tasks
[KAFKA-3187] - Make kafka-acls.sh help messages more generic.
[KAFKA-3188] - Add system test for KIP-31 and KIP-32 - Compatibility Test
[KAFKA-3192] - Add implicit unlimited windowed aggregation for KStream
[KAFKA-3201] - Add system test for KIP-31 and KIP-32 - Upgrade Test
[KAFKA-3202] - Add system test for KIP-31 and KIP-32 - Change message format version on the fly
[KAFKA-3245] - need a way to specify the number of replicas for change log topics
[KAFKA-3276] - Rename 0.10.0.0 to 0.10.1.0 and 0.9.1.0 to 0.10.0.0 in JIRA
[KAFKA-3277] - Update trunk version to be 0.10.0.0-SNAPSHOT
[KAFKA-3286] - Add plugin to quickly check for outdated dependencies
[KAFKA-3289] - Update Kafka protocol guide wiki for KIP-31 / KIP-32
[KAFKA-3306] - Change metadata response to include required additional fields
[KAFKA-3307] - Add ApiVersion request/response and server side handling.
[KAFKA-3311] - Move co-partition checking to PartitionAssignor and auto-create internal topics
[KAFKA-3336] - Unify ser/de pair classes into one serde class
[KAFKA-3337] - Extract selector as a separate groupBy operator for KTable aggregations
[KAFKA-3338] - Add print and writeAsText functions to the Streams DSL
[KAFKA-3361] - Initial protocol documentation page and generation
[KAFKA-3380] - Add system test for GetOffsetShell tool
[KAFKA-3381] - Add system test for SimpleConsumerShell
[KAFKA-3382] - Add system test for ReplicationVerificationTool
[KAFKA-3395] - prefix job id to internal topic names
[KAFKA-3422] - Add overloading functions for each operator not requiring serialization
[KAFKA-3430] - Allow users to set key in KTable.toStream() and KStream
[KAFKA-3439] - Document possible exception thrown in public APIs
[KAFKA-3440] - Add Javadoc for KTable (changelog stream) and KStream (record stream)
[KAFKA-3449] - Rename filterOut() to filterNot() to achieve better terminology
[KAFKA-3451] - Add basic HTML coverage report generation to gradle
[KAFKA-3477] - Add customizable StreamPartition into #to functions of Streams DSL
[KAFKA-3497] - Streams ProcessorContext should support forward() based on child name
[KAFKA-3499] - byte[] should not be used as Map key nor Set member
[KAFKA-3504] - Changelog partition configured to enable log compaction
[KAFKA-3505] - Set curRecord in punctuate() functions
[KAFKA-3508] - Transient failure in kafka.security.auth.SimpleAclAuthorizerTest.testHighConcurrencyModificationOfResourceAcls
[KAFKA-3529] - Transient kafka.api.PlaintextConsumerTest.testAsyncCommit failures
[KAFKA-3598] - Improve JavaDoc of public API
[KAFKA-3599] - Move WindowStoreUtils to package "internals"
[KAFKA-3612] - Add initial set of integration tests
[KAFKA-3662] - Failure in kafka.network.SocketServerTest.tooBigRequestIsRejecte

Bug

[KAFKA-725] - Broker Exception: Attempt to read with a maximum offset less than start offset
[KAFKA-1148] - Delayed fetch/producer requests should be satisfied on a leader change
[KAFKA-1554] - Corrupt index found on clean startup
[KAFKA-1860] - File system errors are not detected unless Kafka tries to write
[KAFKA-2146] - adding partition did not find the correct startIndex
[KAFKA-2157] - kafka-console-consumer.sh: Mismatch in CLI "usage" docs vs. Scala Option parsing
[KAFKA-2181] - online doc issues
[KAFKA-2236] - offset request reply racing with segment rolling
[KAFKA-2399] - Replace Stream.continually with Iterator.continually
[KAFKA-2421] - Upgrade LZ4 to version 1.3 to avoid crashing with IBM Java 7
[KAFKA-2524] - Examples in class javadoc use unimplemented subscribe method
[KAFKA-2551] - Unclean leader election docs outdated
[KAFKA-2578] - Client Metadata internal state should be synchronized
[KAFKA-2589] - Documentation bug: the default value for the "rebalance.backoff.ms" property is not specified correctly
[KAFKA-2672] - SendFailedException when new consumer is run with SSL
[KAFKA-2757] - Consolidate BrokerEndPoint and EndPoint
[KAFKA-2784] - Mirror maker should swallow exceptions during shutdown.
[KAFKA-2803] - Add hard bounce system test for distributed Kafka Connect
[KAFKA-2814] - Kafka Connect system tests using REST interface fail on AWS
[KAFKA-2815] - unit test failure in org.apache.kafka.streams.processor.internals.KafkaStreamingPartitionAssignorTest
[KAFKA-2820] - System tests: log level is no longer propagating from service classes
[KAFKA-2824] - MiniKDC based tests don't run in VirtualBox
[KAFKA-2826] - Make Kafka Connect ducktape services easier to extend
[KAFKA-2844] - Use different keyTab for client and server in SASL tests
[KAFKA-2850] - SslTransportLayerTest.testInvalidEndpointIdentification fails consistently
[KAFKA-2859] - Deadlock in WorkerSourceTask
[KAFKA-2862] - Incorrect help description for MirrorMaker's message.handler.args
[KAFKA-2872] - Error starting KafkaStream caused by sink not being connected to parent source/processor nodes
[KAFKA-2874] - zookeeper-server-stop.sh may fail to shutdown ZK and/or may stop unrelated processes
[KAFKA-2878] - Kafka broker throws OutOfMemory exception with invalid join group request
[KAFKA-2886] - WorkerSinkTask doesn't catch exceptions from rebalance callbacks
[KAFKA-2892] - Consumer Docs Use Wrong Method
[KAFKA-2902] - StreamingConfig getConsumerConfiigs uses getRestoreConsumerConfigs instead of getBaseConsumerConfigs
[KAFKA-2906] - Kafka Connect javadocs not built properly
[KAFKA-2911] - Replace Utils.sleep() with Time.sleep() whenever possible
[KAFKA-2912] - Add error code 4 (InvalidFetchSize) to Errors.java
[KAFKA-2915] - System Tests that use bootstrap.servers embedded in jinja files are not working
[KAFKA-2923] - Improve 0.9.0 Upgrade Documents
[KAFKA-2926] - [MirrorMaker] InternalRebalancer calls wrong method of external rebalancer
[KAFKA-2927] - System tests: reduce storage footprint of collected logs
[KAFKA-2928] - system tests: failures in version-related sanity checks
[KAFKA-2934] - Offset storage file configuration in Connect standalone mode is not included in StandaloneConfig
[KAFKA-2960] - DelayedProduce may cause message loss during repeated leader change
[KAFKA-2965] - Two variables should be exchanged.
[KAFKA-2973] - Fix leak of child sensors on remove
[KAFKA-2990] - NoSuchMethodError when Kafka is compiled with 1.8 and run on 1.7
[KAFKA-2998] - Log warnings when client is disconnected from bootstrap brokers
[KAFKA-2999] - Errors enum should be a 1 to 1 mapping of error codes and exceptions
[KAFKA-3009] - Disallow star imports
[KAFKA-3037] - Number of alive brokers not known after single node cluster startup
[KAFKA-3044] - Consumer.poll doesnot return messages when poll interval is less
[KAFKA-3047] - Explicit offset assignment in Log.append can corrupt the log
[KAFKA-3055] - JsonConverter mangles schema during serialization (fromConnectData)
[KAFKA-3068] - NetworkClient may connect to a different Kafka cluster than originally configured
[KAFKA-3069] - Fix recursion in ZkSecurityMigrator
[KAFKA-3074] - Regex sink connector support
[KAFKA-3080] - ConsoleConsumerTest.test_version system test fails consistently
[KAFKA-3085] - BrokerChangeListener computes inconsistent live/dead broker list
[KAFKA-3088] - 0.9.0.0 broker crash on receipt of produce request with empty client ID
[KAFKA-3091] - Broker with an invalid id would not start when its id is updated to a new valid one
[KAFKA-3095] - No documentation on format of sasl.kerberos.principal.to.local.rules
[KAFKA-3098] - "partition.assignment.strategy" appears twice in documentation
[KAFKA-3112] - One incorrect bootstrap server will prevent Kafka producer from opening
[KAFKA-3132] - URI scheme in "listeners" property should not be case-sensitive
[KAFKA-3134] - Missing required configuration "value.deserializer" when initializing a KafkaConsumer with a valid "valueDeserializer"
[KAFKA-3138] - 0.9.0 docs still say that log compaction doesn't work on compressed topics.
[KAFKA-3140] - PatternSyntaxException thrown in MM, causes MM to hang
[KAFKA-3141] - kafka-acls.sh throws ArrayIndexOutOfBoundsException for an invalid authorizer-property
[KAFKA-3147] - Memory records is not writable in MirrorMaker
[KAFKA-3160] - Kafka LZ4 framing code miscalculates header checksum
[KAFKA-3179] - Kafka consumer delivers message whose offset is earlier than sought offset.
[KAFKA-3189] - Kafka server returns UnknownServerException for inherited exceptions
[KAFKA-3194] - Validate security.inter.broker.protocol against the advertised.listeners protocols
[KAFKA-3195] - Transient test failure in OffsetCheckpointTest.testReadWrite
[KAFKA-3197] - Producer can send message out of order even when in flight request is set to 1.
[KAFKA-3198] - Ticket Renewal Thread exits prematurely due to inverted comparison
[KAFKA-3207] - StateStore seems to be writing state to one topic but restoring from another
[KAFKA-3211] - Handle Connect WorkerTask shutdown before startup correctly
[KAFKA-3216] - "Modifying topics" section incorrectly says you can't change replication factor.
[KAFKA-3217] - Unit tests which dont close producers auto-create topics in Kafka brokers of other tests when port is reused
[KAFKA-3219] - Long topic names mess up broker topic state
[KAFKA-3225] - Method commit() of class SourceTask never invoked
[KAFKA-3229] - Root statestore is not registered with ProcessorStateManager, inner state store is registered instead
[KAFKA-3233] - Per topic consumer metrics missing in new consumer's metrics reporter
[KAFKA-3235] - Unclosed stream in AppInfoParser static block
[KAFKA-3237] - ConfigDef validators require a default value
[KAFKA-3242] - "Add Partition" log message doesn't actually indicate adding a partition
[KAFKA-3243] - Fix Kafka basic ops documentation for Mirror maker, blacklist is not supported for new consumers
[KAFKA-3250] - release tarball is unnecessarily large due to duplicate libraries
[KAFKA-3256] - Large number of system test failures
[KAFKA-3257] - bootstrap-test-env.sh version check fails when grep has --colour option enabled.
[KAFKA-3278] - clientId is not unique in producer/consumer registration leads to mbean warning
[KAFKA-3279] - SASL implementation checks for unused System property java.security.auth.login.config
[KAFKA-3280] - KafkaConsumer Javadoc contains misleading description of heartbeat behavior and correct use
[KAFKA-3285] - Mirror Maker whitelist behavior differs between old and new consumer
[KAFKA-3292] - ClientQuotaManager.getOrCreateQuotaSensors() may return a null ClientSensors.throttleTimeSensor
[KAFKA-3299] - KafkaConnect: DistributedHerder shouldn't wait forever to read configs after rebalance
[KAFKA-3310] - fetch requests can trigger repeated NPE when quota is enabled
[KAFKA-3315] - Add Connect API to expose connector configuration info
[KAFKA-3316] - Add Connect REST API to list available connector classes
[KAFKA-3324] - NullPointerException in StreamPartitionAssignor (Kafka Streams)
[KAFKA-3325] - Out of date instructions in quickstart guide
[KAFKA-3328] - SimpleAclAuthorizer can lose ACLs with frequent add/remove calls
[KAFKA-3330] - Truncate log cleaner offset checkpoint if the log is truncated
[KAFKA-3341] - Improve error handling on invalid requests
[KAFKA-3342] - https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction has log.cleaner.min.cleanable.ratio listed twice in error
[KAFKA-3343] - GroupMetadataManager should use NoTimestamp for message v0
[KAFKA-3344] - Remove previous system test's leftover test-log4j.properties
[KAFKA-3354] - Streams system tests fail in trunk
[KAFKA-3357] - Update to Scala 2.11.8
[KAFKA-3372] - Trailing space in Kafka ConsumerConfig
[KAFKA-3373] - Add `log` prefix to KIP-31/32 configs
[KAFKA-3378] - Client blocks forever if SocketChannel connects instantly
[KAFKA-3384] - bin scripts may not be portable/POSIX compliant
[KAFKA-3388] - Producer should only timeout a batch in the accumulator when metadata is missing.
[KAFKA-3392] - ConsumerRecords iterator throws NoSuchElementException when a TopicPartition is empty
[KAFKA-3394] - Broker fails to parse Null Metadata in OffsetCommit requests
[KAFKA-3397] - use -1(latest) as time default value for tools.GetOffsetShell
[KAFKA-3402] - Restore behaviour of MetadataCache.getTopicMetadata when unsupported security protocol is received
[KAFKA-3409] - Mirror maker hangs indefinitely due to commit
[KAFKA-3412] - Multiple commitAsync() calls causes SendFailedException in commit callback
[KAFKA-3414] - Return of MetadataCache.getAliveBrokers should not be mutated by cache updates
[KAFKA-3421] - Update docs with new connector features
[KAFKA-3424] - Add CORS support to Connect REST API
[KAFKA-3427] - broker can return incorrect version of fetch response when the broker hits an unknown exception
[KAFKA-3434] - Add old ConsumerRecord constructor for compatibility
[KAFKA-3441] - 0.10.0 documentation still says "0.9.0"
[KAFKA-3442] - FetchResponse size exceeds max.partition.fetch.bytes
[KAFKA-3448] - Support zone index in IPv6 regex
[KAFKA-3453] - Transient test failures due to MiniKDC port allocation strategy
[KAFKA-3454] - Add Kafka Streams section in documentation
[KAFKA-3459] - Returning zero task configurations from a connector does not properly clean up existing tasks
[KAFKA-3461] - Fix typos in Kafka web documentations
[KAFKA-3464] - Connect security system tests
[KAFKA-3486] - Autocommit does not execute periodically when partitions are manually assigned
[KAFKA-3488] - commitAsync() fails if metadata update creates new SASL/SSL connection
[KAFKA-3489] - Update request metrics if client closes connection while broker response is in flight
[KAFKA-3510] - OffsetIndex thread safety
[KAFKA-3512] - Add a foreach() operator in Kafka Streams DSL
[KAFKA-3516] - Metrics settings in server.properties cause ClassCastException
[KAFKA-3519] - Refactor Transformer templates to return the same strong-typed value.
[KAFKA-3521] - Better handling NPEs in Streams DSL implementation
[KAFKA-3523] - Capture org.apache.kafka.clients.consumer.CommitFailedException in UncaughtExceptionHandler
[KAFKA-3526] - REST APIs return object representation instead of string for config values, default values and recommended values
[KAFKA-3527] - Consumer commitAsync should not expose internal exceptions
[KAFKA-3528] - AbstractCoordinator wakeup during rebalance can cause spurious rebalances
[KAFKA-3547] - Broker does not disconnect client on unknown request
[KAFKA-3548] - Locale is not handled properly in kafka-consumer
[KAFKA-3557] - Update rocksdb to 4.4.1 and patch updates to snappy and slf4j
[KAFKA-3565] - Producer's throughput lower with compressed data after KIP-31/32
[KAFKA-3569] - commitAsync() sometimes fails with errors
[KAFKA-3581] - Use timeout when joining threads in system test services
[KAFKA-3584] - NullPointer Exception found when Delete Topic and Log delete concurrent
[KAFKA-3587] - LogCleaner fails due to incorrect offset map computation on a replica
[KAFKA-3589] - KTable.count(final KeyValueMapper<K, V, K1> selector, String name) throw NPE
[KAFKA-3594] - Kafka new producer retries doesn't work in 0.9.0.1
[KAFKA-3602] - Rename RecordAccumulator dequeFor() and ensure proper usage
[KAFKA-3605] - Connector REST endpoint allows incorrectly overriding the connector name
[KAFKA-3606] - Traverse CLASSPATH during herder start to list connectors
[KAFKA-3607] - Close state stores explicitly in unit tests upon completing
[KAFKA-3619] - State lock file handle leaks
[KAFKA-3627] - New consumer doesn't run delayed tasks while under load
[KAFKA-3629] - KStreamImpl.to(...) throws NPE when the value SerDe is null
[KAFKA-3631] - Recent addition of "nullable array" to MetadataRequest is breaking MirrorMaker system tests
[KAFKA-3632] - ConsumerLag metrics persist after partition migration
[KAFKA-3633] - Kafka Consumer API breaking backward compatibility
[KAFKA-3639] - Configure default serdes passed via StreamsConfig
[KAFKA-3641] - Fix RecordMetadata constructor backward compatibility
[KAFKA-3642] - Fix NPE from ProcessorStateManager when the changelog topic not exists
[KAFKA-3646] - Console producer using new producer should set timestamp
[KAFKA-3648] - maxTimeToBlock in BufferPool.allocate should be enforced
[KAFKA-3651] - Whenever the BufferPool throws a "Failed to allocate memory within the configured max blocking time" exception, it should also remove the condition object from the waiters deque
[KAFKA-3652] - Return error response for unsupported version of ApiVersionsRequest
[KAFKA-3654] - Config validation should validate both common and connector specific configurations
[KAFKA-3655] - awaitFlushCompletion() in RecordAccumulator should always decrement flushesInProgress count.
[KAFKA-3656] - Avoid stressing system more when already under stress
[KAFKA-3658] - Incorrect validation check on maintenance period with join window size
[KAFKA-3659] - Consumer does not handle coordinator connection blackout period gracefully
[KAFKA-3661] - org.apache.kafka.clients.consumer.RoundRobinAssignor throws NPE when topic metadata not found
[KAFKA-3669] - Add secondary constructor for KafkaConfig with a default value for doLog
[KAFKA-3671] - Topics should not be in common ConnectorConfig definitions
[KAFKA-3673] - Connect tests dont handle concurrent config changes
[KAFKA-3674] - Connector target state changes not propagated to all workers
[KAFKA-3684] - Not all SinkConnectorConfig configs are returned
[KAFKA-3690] - Avoid passing null to UnmodifiableMap
[KAFKA-3692] - Wildcards in External CLASSPATH may cause it not be included in the CLASSPATH
[KAFKA-3694] - Transient system test failure ReplicationTest.test_replication_with_broker_failure.security_protocol
[KAFKA-3713] - test_producer_throughput is failing with compression_type=snappy
[KAFKA-3721] - UpdateMetadataRequest V2 should be in API version 0.10.0-IV1

Improvement

[KAFKA-1215] - Rack-Aware replica assignment option
[KAFKA-2511] - KIP-31 & KIP-32: message format change + adding timestamp to messages
[KAFKA-2547] - Make DynamicConfigManager to use the ZkNodeChangeNotificationListener introduced as part of KAFKA-2211
[KAFKA-2668] - Add a metric that records the total number of metrics
[KAFKA-2800] - Update outdated dependencies
[KAFKA-2816] - selector.poll() no longer throws IOException
[KAFKA-2879] - Make MiniKDC test service slightly more generic
[KAFKA-2881] - Documentation improvement
[KAFKA-2929] - Migrate server side error mapping functionality
[KAFKA-2930] - Update references to ZooKeeper in the docs
[KAFKA-2957] - Fix typos in Kafka documentation
[KAFKA-2958] - Remove duplicate API key mapping functionality
[KAFKA-2964] - Split Security Rolling Upgrade Test By Client and Broker Protocols
[KAFKA-2992] - Trace log statements in the replica fetcher inner loop create large amounts of garbage
[KAFKA-3002] - Make available to specify hostname with Uppercase at broker list
[KAFKA-3006] - Make collection default container type for sequences in the consumer API
[KAFKA-3007] - Implement max.poll.records for new consumer (KIP-41)
[KAFKA-3012] - Avoid reserved.broker.max.id collisions on upgrade
[KAFKA-3013] - Display the topic-partition in the exception message for expired batches in recordAccumulator
[KAFKA-3019] - Add an exceptionName method to Errors
[KAFKA-3024] - Remove old patch review tools
[KAFKA-3029] - Make class org.apache.kafka.common.TopicPartition Serializable
[KAFKA-3043] - Replace request.required.acks with acks in docs
[KAFKA-3058] - remove the usage of deprecated config properties
[KAFKA-3067] - Fix producer API documentation for case when RequiredAcks > 1
[KAFKA-3076] - BrokerChangeListener should log the brokers in order
[KAFKA-3077] - Enable KafkaLog4jAppender to work with SASL enabled brokers.
[KAFKA-3084] - Topic existence checks in topic commands (create, alter, delete)
[KAFKA-3086] - unused handle method in MirrorMakerMessageHandler
[KAFKA-3092] - Rename SinkTask.onPartitionsAssigned/onPartitionsRevoked and Clarify Contract
[KAFKA-3093] - Keep track of connector and task status info, expose it via the REST API
[KAFKA-3105] - Use `Utils.atomicMoveWithFallback` instead of `File.rename`
[KAFKA-3116] - Failure to build
[KAFKA-3119] - Adding -daemon option to zookeeper-server-start.sh USAGE
[KAFKA-3128] - Add metrics for ZooKeeper events
[KAFKA-3149] - Extend SASL implementation to support more mechanisms
[KAFKA-3162] - KIP-42: Add Producer and Consumer Interceptors
[KAFKA-3164] - Document client and mirrormaker upgrade procedure/requirements
[KAFKA-3191] - Improve offset committing JavaDoc in KafkaConsumer
[KAFKA-3196] - KIP-42 (part 2): add record size and CRC to RecordMetadata and ConsumerRecords
[KAFKA-3221] - kafka-acls.sh must verify if a user has sufficient privileges to perform acls CRUD
[KAFKA-3227] - Conservative update of Kafka dependencies
[KAFKA-3253] - Skip duplicate message size check if there is no re-compression during log appending.
[KAFKA-3259] - KIP-31/KIP-32 clean-ups
[KAFKA-3260] - Increase the granularity of commit for SourceTask
[KAFKA-3270] - Add Basic Integration Tests Around ReassignPartitionsCommand
[KAFKA-3272] - Add debugging options to kafka-run-class.sh so we can easily run remote debugging
[KAFKA-3273] - MessageFormatter and MessageReader interfaces should be resilient to changes
[KAFKA-3291] - DumpLogSegment tool should also provide an option to only verify index sanity.
[KAFKA-3301] - CommonClientConfigs.METRICS_SAMPLE_WINDOW_MS_DOC is incorrect
[KAFKA-3303] - Pass partial record metadata to Interceptor onAcknowledgement in case of errors
[KAFKA-3314] - Add CDDL license to LICENSE and NOTICE file
[KAFKA-3318] - Improve consumer rebalance error messaging
[KAFKA-3319] - Improve session timeout broker and client configuration documentation
[KAFKA-3339] - org.apache.kafka.clients.consumer.KafkaConsumer javadoc for seekToBeginning, seekToEnd incomplete
[KAFKA-3352] - Avoid DNS reverse lookups
[KAFKA-3375] - Suppress and fix compiler warnings where reasonable and tweak compiler settings
[KAFKA-3403] - Upgrade ZkClient to 0.8
[KAFKA-3406] - CommonClientConfigs.RETRY_BACKOFF_MS_DOC should be more general
[KAFKA-3407] - ErrorLoggingCallback trims helpful diagnostic information.
[KAFKA-3411] - Streams: stop using "job" terminology, rename job.id to application.id
[KAFKA-3418] - Add section on detecting consumer failures in new consumer javadoc
[KAFKA-3419] - New consumer javadoc unclear on topic subscription vs partition assignment
[KAFKA-3425] - Add missing notes to upgrade docs
[KAFKA-3426] - Improve protocol type errors when invalid sizes are received
[KAFKA-3431] - Remove `o.a.k.common.BrokerEndPoint` in favour of `Node`
[KAFKA-3432] - Cluster.update() thread-safety
[KAFKA-3445] - ConnectorConfig should validate TASKS_MAX_CONFIG's lower bound limit
[KAFKA-3447] - partitionState in UpdateMetadataRequest not logged properly state-change log
[KAFKA-3463] - Change default consumer receive buffer size to 64K
[KAFKA-3470] - Consumer group coordinator should take commit requests as effective as heartbeats
[KAFKA-3475] - Introduce our own `MiniKdc`
[KAFKA-3483] - Restructure ducktape tests to simplify running subsets of tests
[KAFKA-3495] - `NetworkClient.blockingSendAndReceive` should rely on requestTimeout
[KAFKA-3506] - Kafka Connect Task Restart API
[KAFKA-3549] - Close consumers instantiated in consumer tests
[KAFKA-3558] - Add compression_type parameter to benchmarks in benchmark_test.py
[KAFKA-3563] - Maintain MessageAndMetadata constructor compatibility
[KAFKA-3578] - Allow cross origin HTTP requests on all HTTP methods
[KAFKA-3579] - TopicCommand references outdated consumer property fetch.message.max.bytes
[KAFKA-3580] - improve error logging in ReplicaFetchThread with oversized message
[KAFKA-3582] - remove references to Copcyat from connect property files
[KAFKA-3583] - Docs on pause/resume/restart APIs.
[KAFKA-3592] - System tests - don't hardcode paths to scripts
[KAFKA-3611] - Remove WARNs when using reflections
[KAFKA-3613] - Consolidate tumbling windows and hopping windows
[KAFKA-3614] - Consolidate duplicate code in KGroupedTableImpl
[KAFKA-3615] - Exclude test jars in CLASSPATH of kafka-run-class.sh
[KAFKA-3616] - Make kafka producers/consumers injectable for KafkaStreams
[KAFKA-3636] - Increase default group.max.session.timeout.ms to 5 minutes
[KAFKA-3666] - Update new Consumer API links in the docs
[KAFKA-3670] - ControlledShutdownLeaderSelector should pick the preferred replica as the new leader, if possible
[KAFKA-3698] - Update website documentation when it comes to the message format

New Feature

[KAFKA-2832] - support exclude.internal.topics in new consumer
[KAFKA-3046] - add ByteBuffer Serializer&Deserializer
[KAFKA-3490] - Multiple version support for ducktape performance tests

Task

[KAFKA-2830] - Change default fix version to 0.9.1.0 in kafka-merge-pr.py
[KAFKA-2845] - Add 0.9 clients vs 0.8 brokers compatibility test
[KAFKA-2896] - System test for partition re-assignment
[KAFKA-3435] - Remove `Unstable` annotation from new Java Consumer
[KAFKA-3460] - Remove old 0.7 KafkaMigrationTool
[KAFKA-3517] - Document configuration of SASL/PLAIN and multiple mechanisms
[KAFKA-3618] - Handle ApiVersionRequest before SASL handshake
[KAFKA-3621] - Add tests for ApiVersionRequest/Response

Test

[KAFKA-2732] - Add test cases with ZK Auth, SASL and SSL
[KAFKA-2825] - Add controller failover to existing replication tests
[KAFKA-2846] - Add Ducktape test for kafka-consumer-groups
[KAFKA-2905] - System test for rolling upgrade to enable ZooKeeper ACLs with SASL
[KAFKA-2949] - Make EndToEndAuthorizationTest replicated
[KAFKA-2979] - Enable authorizer and ACLs in ducktape tests
[KAFKA-2989] - Verify all partitions consumed after rebalancing in system tests
[KAFKA-3214] - Add consumer system tests for compressed topics
[KAFKA-3255] - Extra unit tests for NetworkClient.connectionDelay(Node node, long now)
[KAFKA-3371] - ClientCompatibilityTest system test failing since KIP-31/KIP-32 was merged
[KAFKA-3520] - System tests of config validate and list connectors REST APIs
[KAFKA-3597] - Enable query ConsoleConsumer and VerifiableProducer if they shutdown cleanly
[KAFKA-3617] - Add unit tests for SASL authentication
[KAFKA-3634] - Add ducktape tests for upgrade with SASL
[KAFKA-3675] - Add lz4 to parametrized `test_upgrade` system test
[KAFKA-3676] - Add system tests for connector pause/resume