Deep Neural Network Technique for High-Dimensional Microwave Modeling and Applications to Parameter Extraction of Microwave Filters

This article introduces the deep neural network method into the field of high-dimensional microwave modeling. Deep learning is nowadays highly successful in solving complex and challenging pattern recognition and classification problems. This article investigates the use of deep neural networks to solve microwave modeling problems that are much more challenging than that solved by the previous shallow neural networks. The most commonly used activation function in the existing deep neural network is the rectified linear unit (ReLU), which is a piecewise hard switch function. However, such a ReLU is not suitable for microwave modeling where the inputoutput relationships are smooth and continuous. In this article, we propose a new deep neural network to perform high-dimensional microwave modeling. A smooth ReLU is proposed for the new deep neural network. The proposed deep neural network employs both the sigmoid function and the smooth ReLU as activation functions. The new deep neural network can represent the smooth inputoutput relationship that is required for microwave modeling. An advanced three-stage deep learning algorithm is proposed to train the new deep neural network model. This algorithm can determine the number of hidden layers with sigmoid functions and those with smooth ReLUs in the training process. It can also overcome the vanishing gradient problem for training the deep neural network. The proposed deep neural network technique can solve microwave modeling problems in a higher dimension than the previous neural network method, i.e., shallow neural network method. Two high-dimensional parameter-extraction modeling examples of microwave filters are presented to demonstrate the proposed deep neural network technique.