Statistical model to estimate a threshold dose and its confidence limits for the analysis of sublinear dose-response relationships, exemplified for mutagenicity data

Strongly sublinear dose-response relationships (slope increasing with dose) raise the question about a putative threshold dose below which no biologically relevant effect would be expected. A mathematical threshold with a break in the curve at the threshold dose is generally rejected for consequences of genotoxicity such as mutation, because proportionality between low dose and the rate of DNA-adduct formation is a reasonable hypothesis. In view of an increasing database for distinct deviation from linearity for mutagenicity. we offer a statistical model to analyze continuous response data and estimate a threshold dose together with its confidence limits, thereby taking data quality and degree of sublinearity into account. The simplest mathematical threshold model is a hockey stick defined by a low-dose part with slope zero at background level a to a theoretical break point at threshold dose td, followed by a linear increase above td with slope b. The function is y (dosed) = a + b x (d - td) x 1([d > td]). Using the free statistics software package R, we make a procedure available to estimate the parameters a, b, and td. Confidence intervals are calculated for all parameters at a significance level that can be defined by the user. If the lower limit of the confidence interval for td is >0. linearity is rejected. The procedure is illustrated by two examples. A small data set with three replicates per dose group, indicating a threshold for the induction of thymidine kinase mutants in L5178Y tk(+/-) mouse lymphoma cells treated with methyl methanesulfonate, did not achieve significance. On the other hand, the large data set reported in this issue (Gocke et al.) on lacZ mutants in bone marrow cells of transgenic mice treated with ethyl methanesulfonate strongly favoured the hockey stick model. The question of a theoretically expected linear dose-related increase below the threshold dose is addressed by linear regression of the data below the break point and estimation of an upper limit of the slope. The question of biological relevance of the resulting slope is discussed against the normal variation of background measures in the control group. (C) 2009 Elsevier B.V. All rights reserved.