SparkContext

1.创建Spark 执行环境SparkEnv

createSparkEnv方法其实是调用createDriverEnv创建SparkEnv,createDriverEnv方法中,获取Driver相关信息,然后调用了SparkEnv私有的create方法。具体请查看SparkEnv

// Create the Spark execution environment (cache, map output tracker, etc)
_env = createSparkEnv(_conf, isLocal, listenerBus)
SparkEnv.set(_env)
  // This function allows components created by SparkEnv to be mocked in unit tests:
  private[spark] def createSparkEnv(
      conf: SparkConf,
      isLocal: Boolean,
      listenerBus: LiveListenerBus): SparkEnv = {
    SparkEnv.createDriverEnv(conf, isLocal, listenerBus, SparkContext.numDriverCores(master))
  }

猜你喜欢

转载自blog.csdn.net/qq_29573903/article/details/82982382