IBM 弄出个DDL，16天缩短到7小时，很暴力# Programming - 葵花宝典
m*n
1 楼
Our software does deep learning training fully synchronously with very low
communication overhead. As a result, when we scaled to a large cluster with
100s of NVIDAI GPUs, it yielded record image recognition accuracy of 33.8%
on 7.5M images from the ImageNet-22k dataset vs the previous best published
result of 29.8% by Microsoft. A 4% increase in accuracy is a big leap
forward; typical improvements in the past have been less than 1%. Our
innovative distributed deep learning (DDL) approach enabled us to not just
improve accuracy, but also to train a ResNet-101 neural network model in
just 7 hours, by leveraging the power of 10s of servers, equipped with 100s
of NVIDIA GPUs; Microsoft took 10 days to train the same model. This
achievement required we create the DDL code and algorithms to overcome
issues inherent to scaling these otherwise powerful deep learning frameworks.
https://www.ibm.com/blogs/research/2017/08/distributed-deep-learning/
communication overhead. As a result, when we scaled to a large cluster with
100s of NVIDAI GPUs, it yielded record image recognition accuracy of 33.8%
on 7.5M images from the ImageNet-22k dataset vs the previous best published
result of 29.8% by Microsoft. A 4% increase in accuracy is a big leap
forward; typical improvements in the past have been less than 1%. Our
innovative distributed deep learning (DDL) approach enabled us to not just
improve accuracy, but also to train a ResNet-101 neural network model in
just 7 hours, by leveraging the power of 10s of servers, equipped with 100s
of NVIDIA GPUs; Microsoft took 10 days to train the same model. This
achievement required we create the DDL code and algorithms to overcome
issues inherent to scaling these otherwise powerful deep learning frameworks.
https://www.ibm.com/blogs/research/2017/08/distributed-deep-learning/
