A novel efficient time and frequency waveform design for filter bank multicarrier communication systems by using Hybrid gray wolf optimization algorithm

To realize reliable high data rate transmission in multicarrier communication systems, the orthogonality of prototype filters in time and frequency is required. However, the time and frequency dispersion of the mobile radio channel leads to the loss of orthogonality which produces intersymbol interference (ISI) and interchannel interference (ICI). Therefore, the prototype filter design plays a critical role in the performance of multicarrier systems. In this paper, we address this issue as an optimization problem. To the best of our knowledge, in all previous prototype filter optimization algorithms, a solution is obtained by working on the samples of filter only in time or frequency domains to satisfy both time and frequency domain characteristics. In this study, the proposed algorithm jointly works on the filter samples both in time and frequency domains. To do this, the hybrid of gray wolf optimization (GWO), Levenberg Marquardt (LM) and pattern search (PS) algorithms is introduced. The GWO algorithm used for exploring space, the LM and PS algorithms exploit the solution, locally. While the PS algorithm nondeterministically changes the samples, the LM algorithm moves all samples by considering gradient descent direction. Furthermore, a smoothing approach is used to achieve better characteristics of designed filter by applying a moving average filter. By using the proposed algorithm, prototype filters can be designed with any possible characteristics based on the user preferences, while in other studies, usually filters are designed with some limited special characteristics. Simulation results show that the prototype filter which is obtained by our proposed algorithm, achieves better characteristics in comparison with the other conventional filters. (C) 2021 Elsevier B.V. All rights reserved.

Automated summary

This paper addresses the issue of prototype filter design in multicarrier communication systems by proposing an algorithm that optimizes filter samples in both time and frequency domains simultaneously. By combining gray wolf optimization, Levenberg Marquardt, and pattern search algorithms, prototype filters with better performance characteristics are designed, outperforming conventional filters in simulation results.