(已解决)执行spark-submit报错ERROR MapOutputTracker:70 - Missing an output location for shuffle 176

问题

执行spark-submit报错ERROR MapOutputTracker:70 - Missing an output location for shuffle 176和Job aborted due to stage failure: ShuffleMapStage 512 (map at MovieLensALS.scala:144) has failed the maximum allowable number of times: 4.
在这里插入图片描述

解决

更改分区:spark-submit命令后加上--conf spark.sql.shuffle.partitions=520,具体数字取决于报错提示
在这里插入图片描述

猜你喜欢

转载自blog.csdn.net/weixin_54218079/article/details/127734235