taskmanager.data.port |
0 |
The task manager’s port used for data exchange operations. |
taskmanager.data.ssl.enabled |
true |
Enable SSL support for the taskmanager data transport. This is applicable only when the global flag for internal SSL (security.ssl.internal.enabled) is set to true |
taskmanager.network.detailed-metrics |
false |
Boolean flag to enable/disable more detailed metrics about inbound/outbound network queue lengths. |
taskmanager.network.memory.buffers-per-channel |
2 |
Maximum number of network buffers to use for each outgoing/incoming channel (subpartition/input channel).In credit-based flow control mode, this indicates how many credits are exclusive in each input channel. It should be configured at least 2 for good performance. 1 buffer is for receiving in-flight data in the subpartition and 1 buffer is for parallel serialization. |
taskmanager.network.memory.floating-buffers-per-gate |
8 |
Number of extra network buffers to use for each outgoing/incoming gate (result partition/input gate). In credit-based flow control mode, this indicates how many floating credits are shared among all the input channels. The floating buffers are distributed based on backlog (real-time output buffers in the subpartition) feedback, and can help relieve back-pressure caused by unbalanced data distribution among the subpartitions. This value should be increased in case of higher round trip times between nodes and/or larger number of machines in the cluster. |
taskmanager.network.memory.fraction |
0.1 |
Fraction of JVM memory to use for network buffers. This determines how many streaming data exchange channels a TaskManager can have at the same time and how well buffered the channels are. If a job is rejected or you get a warning that the system has not enough buffers available, increase this value or the min/max values below. Also note, that "taskmanager.network.memory.min"` and "taskmanager.network.memory.max" may override this fraction. |
taskmanager.network.memory.max |
"1gb" |
Maximum memory size for network buffers. |
taskmanager.network.memory.min |
"64mb" |
Minimum memory size for network buffers. |
taskmanager.network.request-backoff.initial |
100 |
Minimum backoff in milliseconds for partition requests of input channels. |
taskmanager.network.request-backoff.max |
10000 |
Maximum backoff in milliseconds for partition requests of input channels. |