Synthetic Validation of Responses to the NASA Langley Challenge on Optimization under Uncertainty

This paper evaluates responses to the NASA Langley Challenge on Optimization under Uncertainty. The challenge respondents were tasked to quantify the aleatory and epistemic uncertainty impacting the predictions of a computational model calibrated according to a limited number of sub-system and full-system observations. In addition, they evaluated the global sensitivity of the model's prediction with respect to the epistemic variables in order to choose a few of them for refinement. Ultimately, they had to find a reliability-based design that is robust to all/most epistemic realizations within their identified range of uncertainty. The assessments presented herein evaluate the proposed responses within a synthetic validation framework. This framework gauges the ability of the designs to satisfy the reliability requirements imposed upon them against additional data. These data correspond to additional realizations of the aleatory variables, and to the true but unknown value of the epistemic variables. Best practices and lessons learned are set forth based on the comparative analysis of the various responses.

Automated summary

This study evaluates responses to the NASA Langley Challenge on Optimization under Uncertainty, examining how participants quantified uncertainty in computational model predictions and proposed reliability-based design solutions within the identified uncertainty range. The results show that through a synthetic validation framework, designs can be assessed and compared based on reliability requirements imposed by additional data.