Rank order metrics for quantifying the association of sequence features with gene regulation

Motivation: Genome sequences and transcriptome analyses allow the correlation between gene regulation and DNA sequence features to be studied at the whole-genome level. To quantify these correlations, metrics are needed that can be applied to any sequence feature, regardless of its statistical distribution.. It is also desirable for the metric values to be determined objectively, that is, without the use of subjective threshold values. Results: We compare two metrics for quantifying the correlation of DNA sequence features with gene regulation. Each. of the metrics is calculated from a rank-ordering of genes based on the value of the sequence feature of interest. The first metric is the area under the curve for a receiver operator characteristic plot (ROC AUC), a common way of summarizing the tradeoff between sensitivity and specificity for different values of a prediction criterion. We call the second metric the mean normalized conditional probability (MNCP). The MNCP can be thought of as the predictive value of the sequence feature averaged over all regulated genes. The statistical significance (P-value) of each metric can be estimated from simulations. Importantly, the P-value of the MNCP metric is less dramatically affected by the presence of false positives among the set of co-regulated genes than is the ROC AUC. This is especially useful in analyzing gene sets identified by DNA microarray analysis because such data cannot distinguish direct regulation by transcription factor binding from indirect regulation. We demonstrate that these two metrics, taken together, are useful tools for defining the binding site representation and regulatory control regions that best explain the difference between genes that are regulated by a given transcription factor and those that are not. Applications to other gene features are also described. Availability: A Python program for calculating the ROC AUC and MNCP metric values given input rank orders is available from ftp://ftp.bs.jhmi.edu/users/nclarke/MNCP/. Contact: nclarke@jhmi.edu.