运行SparkBench 测试
尝试在单个服务器上进行SparkBench的测试。
测试平台
单台服务器测试
前置条件
了解并安装了Spark软件、SparkBench软件。
测试方法
采用SparkBench开源测试方案，对比目前的集群
生成随机数据
<Workload>/bin/gen_data.sh
运行
<Workload>/bin/run.sh
测试结果
| Apptype (测试方法) | start_ts (开始时间) | duration (运行时间) | size (输入数据大小MB) | throughput (输出数据大小MB) |
|---|---|---|---|---|
| ConnectedComponent-gen | 2018-08-05-17:26:36 | 16.850000 | 88.996429 | 5.281687 |
| ConnectedComponent | 2018-08-05-17:27:10 | 37.094000 | 88.996429 | 2.399213 |
| DecisionTree-gen | 2018-08-05-17:28:30 | 10.579000 | 0.060088 | 0.005679 |
| DecisionTree | 2018-08-05-17:28:54 | 13.434000 | 0.060088 | 0.004472 |
| KMeans | 2018-08-05-17:29:28 | 9.968000 | 0.372898 | 0.037409 |
| LinearRegression-gendata | 2018-08-05-17:31:36 | 10.804000 | 3.924393 | 0.363235 |
| LinearRegression | 2018-08-05-17:32:01 | 18.483000 | 3.924393 | 0.212324 |
| LogisticRegression-gendata | 2018-08-05-17:32:35 | 10.861000 | 7.598749 | 0.699636 |
| LogisticRegression | 2018-08-05-17:33:00 | 17.159000 | 7.598749 | 0.442843 |
| PageRank-gen | 2018-08-05-17:35:56 | 10.345000 | 0.147607 | 0.014268 |
| PageRank | 2018-08-05-17:36:20 | 14.422000 | 0.147607 | 0.010234 |
| PCA-gen | 2018-08-05-17:36:53 | 11.193000 | 18.873979 | 1.686230 |
| PCA | 2018-08-05-17:37:14 | 23.754000 | 18.873979 | 0.794560 |
| PregelOperation-gen | 2018-08-05-17:37:51 | 12.956000 | 0.831987 | 0.064216 |
| PregelOperation | 2018-08-05-17:38:17 | 19.383000 | 0.831987 | 0.042923 |
| ShortestPaths-gen | 2018-08-05-17:38:50 | 26.032000 | 0.279758 | 0.010746 |
| ShortestPaths | 2018-08-05-17:39:30 | 18.323000 | 0.279758 | 0.015268 |
| sql | 2018-08-05-17:40:05 | 2.170000 | 0.056137 | 0.025869 |
| sql_rddRelation | 2018-08-05-17:40:19 | 25.076000 | 0.056137 | 0.002238 |
| StronglyConnectedComponent-gen | 2018-08-05-17:41:30 | 13.650000 | 0.032388 | 0.002372 |
| StronglyConnectedComponent | 2018-08-05-17:41:55 | 73.910000 | 0.032388 | 0.000438 |
| SVDPlusPlus-gen | 2018-08-05-17:43:23 | 15.839000 | 81.302511 | 5.133058 |
| SVDPlusPlus | 2018-08-05-17:43:51 | 97.667000 | 81.302511 | 0.832446 |
| SVM-gen | 2018-08-05-17:46:38 | 12.318000 | 3.924085 | 0.318565 |
| SVM | 2018-08-05-17:47:14 | 25.643000 | 3.924085 | 0.153027 |
| Terasort-gen | 2018-08-05-17:48:37 | 12.516000 | 0.095367 | 0.007619 |
| Terasort | 2018-08-05-17:49:02 | 12.101000 | 0.095367 | 0.007880 |
| TriangleCount-gen | 2018-08-05-17:49:26 | 15.967000 | 12.807078 | 0.802096 |
| TriangleCount | 2018-08-05-17:49:59 | 85.970000 | 12.807078 | 0.148971 |
以上为得到的测试结果，仅供参考！