Hadoop YARN 2.0.4-alpha Release Notes
These release notes include new developer and user-facing incompatibilities, features, and major improvements.
Changes since Hadoop 2.0.3-alpha
- YARN-470.
Major bug reported by Hitesh Shah and fixed by Siddharth Seth (nodemanager)
Support a way to disable resource monitoring on the NodeManager
Currently, the memory management monitor's check is disabled when the maxMem is set to -1. However, the maxMem is also sent to the RM when the NM registers with it ( to define the max limit of allocate-able resources ).
We need an explicit flag to disable monitoring to avoid the problems caused by the overloading of the max memory value.
- YARN-449.
Blocker bug reported by Siddharth Seth and fixed by
HBase test failures when running against Hadoop 2
Post YARN-429, unit tests for HBase continue to fail since the classpath for the MRAppMaster is not being set correctly.
Reverting YARN-129 may fix this, but I'm not sure that's the correct solution. My guess is, as Alexandro pointed out in YARN-129, maven classloader magic is messing up java.class.path.
- YARN-443.
Major improvement reported by Thomas Graves and fixed by Thomas Graves (nodemanager)
allow OS scheduling priority of NM to be different than the containers it launches
It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control.
On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority.
I'm not sure how this applies to windows if at all.
- YARN-429.
Blocker bug reported by Siddharth Seth and fixed by Siddharth Seth (resourcemanager)
capacity-scheduler config missing from yarn-test artifact
MiniYARNCluster and MiniMRCluster are unusable by downstream projects with the 2.0.3-alpha release, since the capacity-scheduler configuration is missing from the test artifact.
hadoop-yarn-server-tests-3.0.0-SNAPSHOT-tests.jar should include the default capacity-scheduler configuration. Also, this doesn't need to be part of the default classpath - and should be moved out of the top level directory in the dist package.