caffe と機械学習

GPU 4個用いて caffe による機械学習の実行中です。

$ nvidia-smi
Fri Aug 19 22:31:58 2016
+------------------------------------------------------+
| NVIDIA-SMI 352.39 Driver Version: 352.39 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K40m Off | 0000:42:00.0 Off | 0 |
| N/A 54C P0 128W / 235W | 4218MiB / 11519MiB | 99% Default |
+-------------------------------+----------------------+----------------------+
| 1 Tesla K40m Off | 0000:43:00.0 Off | 0 |
| N/A 54C P0 132W / 235W | 4219MiB / 11519MiB | 99% Default |
+-------------------------------+----------------------+----------------------+
| 2 Tesla K40m Off | 0000:81:00.0 Off | 0 |
| N/A 52C P0 134W / 235W | 4220MiB / 11519MiB | 99% Default |
+-------------------------------+----------------------+----------------------+
| 3 Tesla K40m Off | 0000:82:00.0 Off | 0 |
| N/A 55C P0 130W / 235W | 4219MiB / 11519MiB | 99% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 47292 C ...tmp/akirat/caffe.new2_5/build/tools/caffe 4192MiB |
| 1 51513 C ...tmp/akirat/caffe.new2_5/build/tools/caffe 4193MiB |
| 2 5111 C ...tmp/akirat/caffe.new2_5/build/tools/caffe 4193MiB |
| 3 56217 C ...tmp/akirat/caffe.new2_5/build/tools/caffe 4192MiB |
+-----------------------------------------------------------------------------+

◯計算サーバ
Huawei RH5885H V3
CPU ：Intel Xeon E7-4890 v2 @ 2.80GHz x 4 socket
Memory ：2.0TB (32GB LRDIMM x 64 DIMMs)
GPU : NVIDIA Tesla K40m x 4
HDD ：2.5-inch 300GB SAS 15000rpm HDD x 2
SSD : ES3000 2.4TB x 2 + 2.5-inch 800GB SSD (Intel DC S3500) x 8
RAID ：RAID-0/1/10/5/50/6/60 1GB Cache with Power Protection
NIC ：On Board 1GE x 4 port interface card
I/O Box ：6 Slot Riser Card x 2、Hot-Plugged Riser Card x 1
PSU ：2000W Platinum AC Power Supply Unit x 2
Rail ：4U Slide Rail with Cable Management Arm
CUDA : 7.5
OS : CentOS 7.2