Release Notes - Apache S4 - Version 0.5.0

S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.

Apache S4 0.5.0 is the first release of S4 as an Apache Incubator project. It is also a complete refactoring of the previous S4 codebase.

Sub-task

• [S4-24] - Dynamic deployment of S4 applications from a remote repository
• [S4-29] - Remove file-based configurations
• [S4-33] - TestUtils and modules should be split across s4-comm and s4-core
• [S4-36] - CommTests should use ZkBasedClusterManagement
• [S4-65] - Create s4 status command

Bug

• [S4-12] - Dynamic Method Dispatching
• [S4-14] - Listener.recv() behavior when interrupted
• [S4-18] - gradle test fails
• [S4-23] - NettyEmitter -- 1. Infinitely waiting on connection, 2. numPartitions is same as numClusterNodes
• [S4-26] - standalone application build script fails when the source directory contains more than 1 class
• [S4-43] - Replace deprecated method in Gradle
• [S4-56] - Prevent concurrent access to published s4r files
• [S4-61] - Threads running the PEs don't have a Context Class Loader
• [S4-63] - onTime() events are not synchronized correctly
• [S4-70] - s4 node command ignores -zk parameter
• [S4-71] - s4 deploy mismanages s4r URI
• [S4-74] - NullPointerException on TCPEmitter.onChange()
• [S4-82] - Fix local communication optimization
• [S4-87] - Checkpointing: recovery : avoid rejections upon fetching
• [S4-89] - Fix references to S4 libs when resolving classpath for adapter in new S4 application

Improvement

• [S4-6] - Netty initialization
• [S4-17] - Put tmp dir under build
• [S4-21] - Create standard PEs, and Event classes
• [S4-22] - Adaptor + inter app communication
• [S4-27] - extensions to cluster configuration through Zookeeper
• [S4-34] - Rename s4.comm.netty and constituents to s4.comm.tcp
• [S4-44] - optional backoff upon multiple consecutive failed checkpoint fetches
• [S4-57] - Document and improve the windowing PE
• [S4-60] - Development mode for rapid application development and testing
• [S4-67] - S4R packaging: improve packaging of dependencies
• [S4-69] - Update gradle to 1.0
• [S4-72] - Resource loading from S4R
• [S4-75] - Code improvements in comm layer
• [S4-78] - Gradle tasks don't finish as soon as possible
• [S4-81] - S4 tools syntax: remove curly braces + expose custom modules config
• [S4-85] - Improve handling of ZK connection changes

New Feature

• [S4-4] - Dynamic App Loading
• [S4-11] - add checkpointing mechanism to s4-piper
• [S4-38] - S4 command line utility

Task

• [S4-31] - Facilities for packaging example and test apps
• [S4-88] - Prepare 0.5.0 release