Tesla C2075 x 8枚で Graph500

前回の続きで、サーバ２台を接続して計算を行ってみました。
Infiniband のボードが古いせいなのか、現在の Infiniband ドライバでは正常に動作しないので、10GbE として使用しています。当然ながら性能低下は起きます。。。


◯ 1サーバ 4GPU : Scale 24
============= Result ==============
SCALE: 24
edgefactor: 16
NBFS: 64
graph_generation: 17.1723642349
num_mpi_processes: 4
construction_time: 20.3061554432
redistribution_time: 1.5020134449
min_time: 0.240659
firstquartile_time: 0.251023
median_time: 0.259705
thirdquartile_time: 0.264707
max_time: 0.277127
mean_time: 0.258595
stddev_time: 0.00918387
min_nedge: 268432547
firstquartile_nedge: 268432547
median_nedge: 268432547
thirdquartile_nedge: 268432547
max_nedge: 268432547
mean_nedge: 268432547
stddev_nedge: 0
min_TEPS: 9.68626e+08
firstquartile_TEPS: 1.01407e+09
median_TEPS: 1.03361e+09
thirdquartile_TEPS: 1.06935e+09
max_TEPS: 1.11541e+09
harmonic_mean_TEPS: 1.03804e+09
harmonic_stddev_TEPS: 4.64464e+06
min_validate: 4.82085
firstquartile_validate: 4.83332
median_validate: 4.83863
thirdquartile_validate: 4.84349
max_validate: 4.85546
mean_validate: 4.83812
stddev_validate: 0.00700683


◯ 2サーバ 8GPU : Scale24
============= Result ==============
SCALE: 24
edgefactor: 16
NBFS: 64
graph_generation: 8.56229615211
num_mpi_processes: 8
construction_time: 13.7278292179
redistribution_time: 2.48683571815
min_time: 0.156456
firstquartile_time: 0.16395
median_time: 0.173233
thirdquartile_time: 0.186077
max_time: 0.394335
mean_time: 0.178429
stddev_time: 0.0299255
min_nedge: 268432547
firstquartile_nedge: 268432547
median_nedge: 268432547
thirdquartile_nedge: 268432547
max_nedge: 268432547
mean_nedge: 268432547
stddev_nedge: 0
min_TEPS: 6.80722e+08
firstquartile_TEPS: 1.44259e+09
median_TEPS: 1.54955e+09
thirdquartile_TEPS: 1.63728e+09
max_TEPS: 1.71571e+09
harmonic_mean_TEPS: 1.50442e+09
harmonic_stddev_TEPS: 3.17888e+07
min_validate: 3.1274
firstquartile_validate: 3.65761
median_validate: 3.87777
thirdquartile_validate: 4.15296
max_validate: 5.54566
mean_validate: 3.94297
stddev_validate: 0.450207


◯ 2サーバ 8GPU : Scale25
============= Result ==============
SCALE: 25
edgefactor: 16
NBFS: 64
graph_generation: 17.3240265846
num_mpi_processes: 8
construction_time: 28.9375069141
redistribution_time: 4.77809858322
min_time: 0.30046
firstquartile_time: 0.313659
median_time: 0.328601
thirdquartile_time: 0.341993
max_time: 0.746691
mean_time: 0.338517
stddev_time: 0.0612153
min_nedge: 536865258
firstquartile_nedge: 536865258
median_nedge: 536865258
thirdquartile_nedge: 536865258
max_nedge: 536865258
mean_nedge: 536865258
stddev_nedge: 0
min_TEPS: 7.18992e+08
firstquartile_TEPS: 1.56981e+09
median_TEPS: 1.63379e+09
thirdquartile_TEPS: 1.71162e+09
max_TEPS: 1.78681e+09
harmonic_mean_TEPS: 1.58593e+09
harmonic_stddev_TEPS: 3.61322e+07
min_validate: 7.04169
firstquartile_validate: 7.63266
median_validate: 7.95585
thirdquartile_validate: 8.45642
max_validate: 9.45792
mean_validate: 8.04152
stddev_validate: 0.547952


◯ 2サーバ 8GPU : Scale26
============= Result ==============
SCALE: 26
edgefactor: 16
NBFS: 64
graph_generation: 35.3478515148
num_mpi_processes: 8
construction_time: 62.205946207
redistribution_time: 9.73122549057
min_time: 0.601114
firstquartile_time: 0.613692
median_time: 0.622049
thirdquartile_time: 0.638401
max_time: 1.0013
mean_time: 0.63633
stddev_time: 0.0561822
min_nedge: 1073731075
firstquartile_nedge: 1073731075
median_nedge: 1073731075
thirdquartile_nedge: 1073731075
max_nedge: 1073731075
mean_nedge: 1073731075
stddev_nedge: 0
min_TEPS: 1.07233e+09
firstquartile_TEPS: 1.68191e+09
median_TEPS: 1.72612e+09
thirdquartile_TEPS: 1.74962e+09
max_TEPS: 1.78624e+09
harmonic_mean_TEPS: 1.68738e+09
harmonic_stddev_TEPS: 1.87698e+07
min_validate: 14.0408
firstquartile_validate: 14.9233
median_validate: 15.5739
thirdquartile_validate: 16.1965
max_validate: 17.8704
mean_validate: 15.5851
stddev_validate: 0.795499


サーバの仕様
Intel Xeon + 4 GPU マシン
CPU：Xeon X5690(3.46GHz,6core)×2
Memory：192GB(16GB×12)
HDD：SATA500GB×2
NIC : GbE x 1 & Inifiniband(FDR) x 1
GPGPU：Tesla C2075×4
OS：CentOS 7.5