Face, Construct and Criterion Validity, and Test-Retest Reliability, of the Adult Rejection Sensitivity Questionnaire

This research sought to test the face, construct and criterion validity, and test-retest reliability of the Adult Rejection Sensitivity Questionnaire (ARSQ). In Study 1, participants (n = 45) completed the ARSQ and questions assessing scale item relevancy, clarity, difficulty, and sensitivity. In Study 2, participants (n = 513) completed the ARSQ and demographic questions. In Study 3, participants (n = 244) completed the ARSQ and returned 2 weeks later to complete the ARSQ and measures of depression, anxiety, and self-silencing behavior. Study 1 provided strong support for face validity with all items deemed relevant, clear, easy to answer, and neither distressing nor judgmental. Study 2 provided adequate support for the factor structure of the ARSQ (single-factor model and two-factor model) but suggested modifications could be made to improve scale validity. Study 3 provided further support for an adequate (but not good) factor structure, and evidence for criterion validity established through medium-large effect size correlations with depression, anxiety, and self-silencing behavior. However, the 2-week scale stability appeared poor (r = .45) in a subsample of participants. Overall, the ARSQ showed sufficient validity to recommend its continued use, but we recommend further tests of scale reliability and potential modifications to increase construct validity.

Automated summary

This research aimed to assess the validity and reliability of the Adult Rejection Sensitivity Questionnaire (ARSQ) through multiple studies. Study 1 showed strong face validity with all items considered relevant, clear, easy to answer, and non-distressing. Study 2 supported the factor structure of the ARSQ but suggested potential improvements. Study 3 provided further support for the factor structure and demonstrated criterion validity through correlations with depression, anxiety, and self-silencing behavior. However, the 2-week scale stability was poor in a subset of participants. Overall, the ARSQ showed sufficient validity, but further tests and modifications are recommended for increased construct validity.