The rapid growth of high performance supercomputing technology and advances in numerical techniques in the last two decades have provided an unprecedented opportunity to explore complex physical phenomena using modeling and simulation. Today, multiphysics simulations involving fluid flow, structural dynamics, chemical kinetics, and atomic and nuclear sciences are ubiquitous. The success of computer simulations also means that they will be increasingly relied upon as important tools for high-consequence predictions and decision making. For example, the Department of Energy Advanced Simulation and Computing (ASC) Program was established in 1995 to accelerate the development of computer simulation capabilities for analyzing and predicting the performance, safety, and reliability of nuclear weapons and certifying their functionality. As such, “predictive science,” which is the application of verified and validated computational simulations to predict the behavior of complex systems, has emerged as a new movement.
The flourishing of simulation-based scientific discovery has also resulted in the emergence of the verification and validation (V&V) and uncertainty quantification (UQ) disciplines. The goal of these emerging disciplines is to enable scientists to make precise statements about the degree of confidence they have in their simulation-based predictions. Here we focus on the UQ discipline, which is essential for validating and verifying computer models.
According to the 2012 report by the National Research Council Committee on the Mathematical Foundations of Verification, Validation, and Uncertainty Quantification, UQ is defined as “the process of quantifying uncertainties associated with model calculations of true, physical quantities of interest, with the goals of accounting for all sources of uncertainty and quantifying the contributions of specific sources to the overall uncertainty.”
A UQ methodology generally consists of the following tasks:
- Identify all sources of uncertainty—Where are the uncertainties?
- Characterize the identified sources of uncertainty—What form they are in?
- Propagate the uncertainties through the model—How do they evolve during the simulation?
- Analyze the effect of uncertainties on the quantities of interest—What are their impacts?
- Reduce the model uncertainties if deemed necessary.
In mapping from reality to computer models, many sources of uncertainties/discrepancies arise:
- Uncertainties due to variabilities in the actual design parameter values and environmental factors
- Uncertainties in initial and boundary conditions
- Uncertainties in the physics sub-models (due to imprecise and simplified physics, e.g., data-driven phenomenological models)
- Uncertainties in couplings between sub-models
- Uncertainties due to missing physics
- Uncertainties arising from model implementation, discretization errors, roundoff errors, and algorithmic errors
- Uncertainties in data due to noise and measurement errors
- Uncertainties due to lack of sufficient data
Uncertainties are often classified as:
- aleatoric (known probability distributions)
- epistemic (probability distributions unknown, use intervals/belief functions)
- mixed aleatoric/epistemic (known distribution, unknown means/standard deviations)
- model-form uncertainties (each possible model may have its own aleatoric/epistemic uncertainties)
Characterization of uncertainties is often the most time-consuming part of a UQ study, especially for data-driven phenomenological physics models.
Propagation of uncertainties can be performed forward and backward involving
- analyzing the impact parameter uncertainties have on model outputs,
- finding the major sources of uncertainties (sensitivity analysis),
- deriving parameter posterior distributions based on data (calibration/data fusion), and
- exploring “interesting” regions in the parameter space (model exploration).
Approaches for propagating uncertainties can fall into three areas:
- intrusive (polynomial chaos, interval method, DES-UQ)
- non-intrusive/semi-intrusive (sampling-based, PSUADE, DASSI)
- hybrid (a mixture of both intrusive and non-intrusive methods inside a simulation, MEDUSA)
Selecting methods for propagating uncertainties is a critical part of planning. Care must be taken to ensure that proper methods are selected for a study. It is useful to first identify UQ characteristics of a given model beforehand. For example, for a complex multiphysics model, we may extract the following characteristics:
- Nonlinear relationship between the uncertain and output variables
- The uncertain parameter space is high-dimensional
- There may be some model form uncertainties
- High computational cost per simulation
- Experimental data are available at module, subsystem, and full system level
Knowing these characteristics will help determine the best UQ strategies. For example, if there are many uncertain parameters, then reducing the parameter dimension may be a useful step before more detailed analysis. If the mode input-output relationships are nonlinear, then perturbation-based or SRC (standardized regression coefficient)-based sensitivity analysis may not be sufficient. For computationally expensive models, response surfaces may be needed for quantitative analysis. If the model is highly nonlinear, classical regression-based response surface methods may not be good enough (non-parametric methods may be needed).
Information gathered from simulations can be analyzed for, for example,
- assessing “anomalous” regions in the parameter space (risk analysis)
- establishing the integrity of a simulation model (validation)
- providing information on which additional physical experiments are needed to best improve the understanding of the system (experimental guidance)
The planning of a UQ study involves three components:
1. Develop a UQ Process
- Define objectives of the study (e.g., global sensitivity analysis)
- Define clearly and in detail the model to be studied (e.g., inputs, outputs, model characteristics, software version, physics assumption)
- Determine the computational budget
- Step through the UQ methodology: identification, characterization, …
2. Compile relevant UQ methods and tools such as those for
- Dimension reduction (to reduce the number of model parameters)
- Response surface analysis
- Uncertainty assessment/sensitivity analysis
- Parameter inference
- Model validation
3. Develop/secure hardware/software infrastructure to perform ensemble runs
- Job management: scheduling, monitoring
- Data management, visualization
Navigate to Software page