Expected Information Gain (EIG) Optimality Criteria#

Module providing implementations of the EIG (expected information gain) OED criterion for Bayesian inverse problems involving model-constrained inverse problems.

class EvaluationMethod(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#

Bases: StrEnum

NMC = '\\Anmc\\Z'#
class BayesianInversionEIGConfigs(*, debug=False, verbose=False, output_dir='./_PYOED_RESULTS_', oed_problem=None, lp_behavior=LP_Behavior.MAP, name='EIG: Expected Information Gain utility function for Bayesian inverse problems', evaluation_method='nmc', inner_sample_size=256, outer_sample_size=256, adaptive_contrastive_estimation=False, laplace_importance_sampling=False, random_seed=None)[source]#

Bases: BayesianInversionCriterionConfigs

Configuration class for Bayesian EIG criterion provided by

Parameters:

  • verbose (bool) – a boolean flag to control verbosity of the object.

  • debug (bool) – a boolean flag that enables adding extra functionlity in a debug mode

  • output_dir (str | Path) – the base directory where the output files will be saved.

  • oed_problem (InversionOED | None) – OED problem for which the utility function is defined. This must be an instance of InversionOED.

  • lp_behavior (LP_Behavior | str) – behavior of the linearization point (MAP, prior, manual). By default, it is set to MAP. However, if the oed problem is linear, the passed parameter is ignored and is changed to LINEAR instead. If you wish to forcefully choose the linearization point despite the linearity of the problem, set the field after instantiation.

  • evaluation_method (str) –

    name ofthe method to use for computing the criterion and its gradients. Supported are:

    • "nmc": Estimate the criterion using a nested Monte Carlo

      scheme.

  • inner_sample_size (int) – number of sample points for the inner loop of the double loop Monte-Carlo (MC) estimator.

  • outer_sample_size (int) – number of sample points for the outer loop of the double loop Monte-Carlo (MC) estimator.

  • adaptive_contrastive_estimation (bool) – whether to apply adaptive contrastive estimation or not.

  • laplace_importance_sampling (bool) – whether to carry out Laplace importance sampling for the estimate or not.

  • random_seed (None | int) – The random seed to use for randomized algorithms.

name: str#
evaluation_method: str#
inner_sample_size: int#
outer_sample_size: int#
adaptive_contrastive_estimation: bool#
laplace_importance_sampling: bool#
random_seed: None | int#
__init__(*, debug=False, verbose=False, output_dir='./_PYOED_RESULTS_', oed_problem=None, lp_behavior=LP_Behavior.MAP, name='EIG: Expected Information Gain utility function for Bayesian inverse problems', evaluation_method='nmc', inner_sample_size=256, outer_sample_size=256, adaptive_contrastive_estimation=False, laplace_importance_sampling=False, random_seed=None)#
class BayesianInversionEIG(configs=None)[source]#

Bases: BayesianInversionCriterion

An implementation of the Expected Information Gain (EIG) criterion for Bayesian Optimal Design of Experiments. The criterion is defined as the expectation of the KL divergence over the data, i.e.

\[\mathcal{U}(\zeta) = \int_\mathcal{Y} p(\mathbf{y} | \zeta) \left( \int_\Theta p(\theta | \mathbf{y}, \zeta) \log \frac{ p(\theta | \mathbf{y}, \zeta) }{p(\theta)} d\theta \right) d\mathbf{y}\]

where \(\zeta\) is the design, \(\mathbf{y}\) is the data with associated space \(\mathcal{Y}\), and \(\theta\) is the inference parameter with associated space \(\Theta\). Likewise, \(p(\theta)\) is the prior density, \(p(\theta | \mathbf{y}, \zeta)\) is the posterior, and \(p(\mathbf{y} | \zeta)\) is the marginal density of the observed data, i.e. \(p(\mathbf{y} | \zeta) = \mathbb{E}_{\theta \sim p(\theta)} [p(\mathbf{y} | \theta, \zeta)]\).

Warning

Currently laplace_importance_sampling does not make any difference. It’s effect will be implemented.

__init__(configs=None)[source]#

Constructor for utility functions / optimality criteria.

Parameters:

configs (dict) – Object holding configurations for the criterion.

update_evaluation_method(evaluation_method)[source]#

Update the evaluation method.

validate_configurations(configs, raise_for_invalid=True)[source]#

A method to check the passed configurations and make sure they are conformable with each other, and with current configurations (or default of not set)

Parameters:

  • configs (dict | BayesianInversionEIGConfigs) – an object holding key/value configurations

  • raise_for_invalid (bool) – if True raise TypeError for invalid configrations key

Returns:

True/False flag indicating whether passed configurations is valid or not

update_configurations(**kwargs)[source]#

Update one or more configurations given their name and value. Each configuration key/value is validated against current values (more precedence to passed, then current settings).

If all passed configurations are conformable with each other, and with current configurations, the passed configurations updated the underlying configurations dictionary.

evaluate(design)[source]#

Evaluate the EIG for the given OED problem at the given design.

Parameters:

design – an observational design vector conformable with the observation operator and the observation error covariance matrix operator.

Note

If you are intending to compare the EIG against different designs, it is recommended to set the random seed to a fixed value to ensure consistency amongst the internal sampling.

Returns:

the EIG value at the given design.

Return type:

float

evaluate_nmc_estimator(prior_sampler, data_sampler, likelihood, outer_sample_size=<member 'outer_sample_size' of 'BayesianInversionEIGConfigs' objects>, inner_sample_size=<member 'inner_sample_size' of 'BayesianInversionEIGConfigs' objects>, adaptive_contrastive_estimation=<member 'adaptive_contrastive_estimation' of 'BayesianInversionEIGConfigs' objects>)[source]#

Estimate the EIG using a nested Monte Carlo scheme. It is given by

\[\hat{\mathcal{U}}_{\rm NMC}(\zeta) = \frac{1}{N} \sum_{i=1}^N \left[ \log(p(\mathbf{y}_n | \theta_{n,0}, \zeta)) - \log \left( \frac{1}{M} \sum_{m=1}^M p(\mathbf{y}_n | \theta_{n,m}, \zeta) \right) \right]\]

where \(\theta_{n,m} \sim p(\theta)\) and \(\mathbf{y}_n \sim p(\mathbf{y} | \theta_{n,0}, \zeta)\). The parameters \(outer_sample_size\) and \(inner_sample_size\) are set by the based on corresponding configurations attribute. If wish to change them, refer to Rainforth et. al [1] for additional reading, but briefly speaking, it is recommended to select \(outer_sample_size\) and \(inner_sample_size\) such that \(outer_sample_size \approx inner_sample_size^2\).

Parameters:

  • prior_sampler (callable) – a function that returns a sample from the prior.

  • data_sampler (callable) – a function that returns a sample from the data given a parameter sample.

  • likelihood (callable) – a function that returns the likelihood of a data sample given a parameter sample.

Note

Note, assuming likelihood is normally distributed, the normalization factor cancels out in the above expression. Hence, one can save computational resources by not normalizing the likelihood.

Parameters:

  • outer_sample_size (int) – the number of outer samples.

  • inner_sample_size (int) – the number of inner samples.

  • adaptive_contrastive_estimation (bool) – whether to use the Adaptive contrastive Estimation or not. This manifests as using \(\theta_{n,0}\) inside the inner Monte Carlo. This can be useful if the prior is not a good approximation of the posterior, as this may manifest as many prior samples missing the ‘support’ of the likelihood, resulting in infinities in the log-likelihood. Note, this changes NMC into a lower bound of the EIG. Refer to Foster et. al [2] for additional reading. Default is True.

Returns:

An estimation of the EIG.

Return type:

float

property evaluation_method#

The method used for computing the criterion value and its gradients. Two options:

  • ‘nmc’: Nested monte carlo estimation

property inner_sample_size#

Return the sample size used in the inner loop of the double loop MC estimate.

property outer_sample_size#

Return the sample size used in the outer loop of the double loop MC estimate.

property adaptive_contrastive_estimation#

Return the flag indicating whether to use adaptive contrastive estimation

property laplace_importance_sampling#

Return the flag indicating whether to use laplace importance sampling

property random_seed#

Return the random seed used for randomized algorithms