Pure intra mode decision in HEVC using optimized firefly algorithm

High Efficiency Video Coding (HEVC) is the latest encoder that increased the intra modes from 9 to 35 to efficiently handle the contents of the video. The HEVC's test model (HM) selects the optimal intra mode using the brute force method which increases the complexity of HEVC. This work, firstly, investigates the feasibility of firefly algorithm (FFA) due to its exploration and exploitation characteristics to expedite the intra mode decision in HEVC. Secondly, a novel objective function is formulated for FFA to efficiently compute the brightness for intra modes in FFA. Thirdly, the parameters of FFA are made dynamic to adjust according to the contents of video sequences. Simulation results demonstrate that the nature inspired algorithm, FFA, pays off by saving a minimum of 27% of the total coding time on average and doesn't sacrifice quality by limiting Bjontegaard delta bit rate (BD-BR) increase to only 0.98% on average. (C) 2020 Elsevier Inc. All rights reserved.