期刊
IEEE ACCESS
卷 8, 期 -, 页码 211860-211868出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/ACCESS.2020.3039833
关键词
Deep learning; Training; Libraries; Computer architecture; Software; Computational modeling; Microprocessors; Deep learning; Pytorch; torch; Keras; TensorFlow; reproducibility; reproducible; repeatability; replicability; replicable deep learning models; deterministic models; determinism
资金
- National Science Foundation [1746511, 1926990, 1513126]
- Direct For Computer & Info Scie & Enginr
- Division Of Computer and Network Systems [1513126] Funding Source: National Science Foundation
- Div Of Industrial Innovation & Partnersh
- Directorate For Engineering [1926990] Funding Source: National Science Foundation
- Div Of Industrial Innovation & Partnersh
- Directorate For Engineering [1746511] Funding Source: National Science Foundation
Deep learning training typically starts with a random sampling initialization approach to set the weights of trainable layers. Therefore, different and/or uncontrolled weight initialization prevents learning the same model multiple times. Consequently, such models yield different results during testing. However, even with the exact same initialization for the weights, a lack of repeatability, replicability, and reproducibility may still be observed during deep learning for many reasons such as software versions, implementation variations, and hardware differences. In this article, we study repeatability when training deep learning models for segmentation and classification tasks using U-Net and LeNet-5 architectures in two development environments Pytorch and Keras (with TensorFlow backend). We show that even with the available control of randomization in Keras and TensorFlow, there are uncontrolled randomizations. We also show repeatable results for the same deep learning architectures using the Pytorch deep learning library. Finally, we discuss variations in the implementation of the weight initialization algorithm across deep learning libraries as a source of uncontrolled error in deep learning results.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据