Spark executor内存配置参数优先级 2015-08-11 19:30

executor内存参数优先级

1
2
3
4
5
val executorMemory = conf.getOption("spark.executor.memory")
        .orElse(Option(System.getenv("SPARK_EXECUTOR_MEMORY")))
        .orElse(Option(System.getenv("SPARK_MEM")).map(warnSparkMem))
    .map(Utils.memoryStringToMb)
    .getOrElse(512)
Tags: #Spark    Post on Spark