Cosmological test of gravity with polarizations of stochastic gravitational waves around 0.1-1 Hz

In general relativity, a gravitational wave has two polarization modes (tensor mode), but it could have additional polarizations (scalar and vector modes) in the early stage of the Universe, where the general relativity may not strictly hold and/or the effect of higher-dimensional gravity may become significant. In this paper, we discuss how to detect extra-polarization modes of stochastic gravitational wave background (GWB), and study the separability of each polarization using future space-based detectors such as BBO and DECIGO. We specifically consider two plausible setups of the spacecraft constellations consisting of two and four clusters, and estimate the sensitivity to each polarization mode of GWBs. We find that a separate detection of each polarization mode is rather sensitive to the geometric configuration and distance between clusters and that the clusters should be, in general, separated by an appropriate distance. This seriously degrades the signal sensitivity; however, for suitable conditions, the space-based detector can separately detect scalar, vector and tensor modes of GWBs with energy density as low as h(0)(2)Omega(gw) similar to 10(-15).