Orbital engineering of two-dimensional materials with hydrogenation: A realization of giant gap and strongly correlated topological insulators

Orbital interaction plays an important role in topological insulators. Using first-principles calculations, we demonstrate that hydrogenation can change two-dimensional (2D) trivial insulator/semimetal-like Pb, Mo, and W to a Z(2) topological insulator with giant gaps by filtering unwanted orbitals such as the p(z) orbital in Pb, and the d(3z2-r2) in Mo andW. For PbH, the large intrinsic spin-orbit coupling (SOC) confined in the in-plane orbitals p(x,y) results in a bulk gap of 1.07 eV. For the case of MoH andWH, hydrogenation results in a novel sd(4) hybridization with a Dirac cone formed by the out-of-plane orbital d(yz/xz) orbitals. Furthermore, due to the electron-electron interaction and the strong SOC in the 4d and 5d elements, the bulk band gap is significantly enhanced with a value of 1.28 eV for MoH and 0.30 eV for WH. The strong correlation effect also induces several topological phase transitions in WH like the Dirac semimetal and Mott insulating phase, while MoH remains a topological insulator even with large correlation effect. We propose that these new strongly correlated 2D materials can be realized experimentally by using substrate such as the boron nitride sheet, which can exhibit various novel properties such as the Dirac semimetal phase with large spin-orbit energy splitting as well as the quantum spin Hall property.