Flink parallelism provided

The task parallelism can be specified at different levels of Flink. Four levels are: operator level, the execution environment (ExecutionEnvironment) level, the client (command line) level, the configuration file (flink-conf.yaml) level

* Each operator, data source or data sink can by calling setParallelism()to the specified method

* The default number of concurrent operating environment by calling setParallelism()to the specified method.env.setParallelism(3);运行环境的并发数可以被每个算子确切的并发数配置所覆盖。

* 对于CLI客户端,并发参数可以通过-p来指定

* 影响所有运行环境的系统级别的默认并发度可以在./conf/flink-conf.yamlparallelism.defaul项中指定。不建议


当然,你也可以设置最大的并行度

* 你可以通过调用setMaxParallelism()方法来设置最大并发度。


Flink how to determine the number TaskManager? Answer: The maximum degree of parallelism Job divided by the number assigned to each task slot TaskManager

Flink on YARN, the number TaskManager is: max(parallelism) / yarnslots(rounded up). For example, a maximum degree of parallelism of 10, each job has two tasks TaskManager groove 5 will start TaskManager




reference:

https://cloud.tencent.com/developer/article/1500184

https://www.jianshu.com/p/9c301ff1a9b2

Guess you like

Origin www.cnblogs.com/maoxiangyi/p/11856045.html