Release Notes - Kafka - Version 2.0.1

Below is a summary of the JIRA issues addressed in the 2.0.1 release of Kafka. For full documentation of the release, a guide to get started, and information about the project, see the Kafka project site.

Note about upgrades: Please carefully review the upgrade documentation for this release thoroughly before upgrading your cluster. The upgrade notes discuss any critical information about incompatibilities and breaking changes, performance changes, and any other changes that might impact your production deployment of Kafka.

The documentation for the most recent release can be found at http://kafka.apache.org/documentation.html.

Improvement

• [KAFKA-6914] - Kafka Connect - Plugins class should have a constructor that can take in parent ClassLoader
• [KAFKA-7131] - Update release script to generate announcement email text
• [KAFKA-7198] - Enhance KafkaStreams start method javadoc
• [KAFKA-7278] - replaceSegments() should not call asyncDeleteSegment() for segments which have been removed from segments list
• [KAFKA-7285] - Streams should be more fencing-sensitive during task suspension under EOS
• [KAFKA-7414] - Do not fail broker on out of range offsets in replica fetcher
• [KAFKA-7454] - Use lazy allocation for SslTransportLayer buffers

Bug

• [KAFKA-4950] - ConcurrentModificationException when iterating over Kafka Metrics
• [KAFKA-5891] - Cast transformation fails if record schema contains timestamp field
• [KAFKA-6648] - Fetcher.getTopicMetadata() should return all partitions for each requested topic
• [KAFKA-7044] - kafka-consumer-groups.sh NullPointerException describing round robin or sticky assignors
• [KAFKA-7119] - Intermittent test failure with GSSAPI authentication failure
• [KAFKA-7128] - Lagging high watermark can lead to committed data loss after ISR expansion
• [KAFKA-7144] - Kafka Streams doesn't properly balance partition assignment
• [KAFKA-7158] - Duplicates when searching kafka stream state store with caching
• [KAFKA-7164] - Follower should truncate after every leader epoch change
• [KAFKA-7180] - In testHWCheckpointWithFailuresSingleLogSegment, wait until server1 has joined the ISR before shutting down server2
• [KAFKA-7192] - State-store can desynchronise with changelog
• [KAFKA-7196] - Remove heartbeat delayed operation for those removed consumers at the end of each rebalance
• [KAFKA-7216] - Exception while running kafka-acls.sh from 1.0 env on target Kafka env with 1.1.1
• [KAFKA-7225] - Kafka Connect ConfigProvider not invoked before validation
• [KAFKA-7242] - Externalized secrets are revealed in task configuration
• [KAFKA-7250] - Kafka-Streams-Scala DSL transform shares transformer instance
• [KAFKA-7255] - Timing issue in SimpleAclAuthorizer with concurrent create/update
• [KAFKA-7261] - Request and response total metrics record bytes instead of request count
• [KAFKA-7280] - ConcurrentModificationException in FetchSessionHandler in heartbeat thread
• [KAFKA-7284] - Producer getting fenced may cause Streams to shut down
• [KAFKA-7287] - Set open ACL permissions for old consumer znode path
• [KAFKA-7296] - Producer should handle COORDINATOR_LOADING error in TxnOffsetCommit
• [KAFKA-7298] - Concurrent DeleteRecords can lead to fatal OutOfSequence error in producer
• [KAFKA-7301] - KTable to KTable join invocation does not resolve in Scala DSL
• [KAFKA-7316] - Use of filter method in KTable.scala may result in StackOverflowError
• [KAFKA-7353] - Connect logs 'this' for anonymous inner classes
• [KAFKA-7354] - Fix IdlePercent and NetworkProcessorAvgIdlePercent metric calculation
• [KAFKA-7369] - Retry when possible in AdminClient.listConsumerGroups
• [KAFKA-7385] - Log cleaner crashes when empty batches are retained with idempotent or transactional producers
• [KAFKA-7386] - Streams Scala wrapper should not cache serdes
• [KAFKA-7415] - OffsetsForLeaderEpoch may incorrectly respond with undefined epoch causing truncation to HW
• [KAFKA-7434] - DeadLetterQueueReporter throws NPE if transform throws NPE
• [KAFKA-7453] - Enable idle expiry of connections which are never selected
• [KAFKA-7459] - Concurrency bug in updating RequestsPerSec metric
• [KAFKA-7460] - Connect Values converter uses incorrect date format string
• [KAFKA-7464] - Fail to shutdown ReplicaManager during broker cleaned shutdown
• [KAFKA-7467] - NoSuchElementException is raised because controlBatch is empty
• [KAFKA-7476] - SchemaProjector is not properly handling Date-based logical types
• [KAFKA-7519] - Transactional Ids Left in Pending State by TransactionStateManager During Transactional Id Expiration Are Unusable
• [KAFKA-7535] - KafkaConsumer doesn't report records-lag if isolation.level is read_committed

Task

• [KAFKA-7228] - DeadLetterQueue throws a NullPointerException

Sub-task

• [KAFKA-7299] - batch LeaderAndIsr requests during auto preferred leader election