The Brezis-Nirenberg problem for the curl-curl operator

We look for solutions E : Omega -> R-3 of the problem {del x (del x E) + lambda E = vertical bar E vertical bar(p-2) E in Omega v x E=0 on partial derivative Omega on a bounded Lipschitz domain Omega subset of R-3, where del x denotes the curl operator in R-3. The equation describes the propagation of the time-harmonic electric field R{E(x)e(iwt)} in a nonlinear isotropic material Omega with lambda = -mu epsilon omega(2) <= 0, where mu and epsilon stand for the permeability and the linear part of the permittivity of the material. The nonlinear term vertical bar E vertical bar(p-2) E with p > 2 is responsible for the nonlinear polarisation of Omega and the boundary conditions are those for Omega surrounded by a perfect conductor. The problem has a variational structure and we deal with the critical value p, for instance, in convex domains Omega or in domains with C-1,C-1 boundary, p = 6 = 2* is the Sobolev critical exponent and we get the quintic nonlinearity in the equation. We show that there exist a cylindrically symmetric ground state solution and a finite number of cylindrically symmetric bound states depending on lambda <= 0. We develop a new critical point theory which allows to solve the problem, and which enables us to treat more general anisotropic media as well as other variational problems. (C) 2018 Elsevier Inc. All rights reserved.