Data Assimilation (Inference) Examples#

Three-dimensional variational (3DVar) data assimilation#

Driver script to test the components of 4DVar data assimilation system

Functions

main(test)

Choose the test (4D-var data assimilation for a selected simulation mdoel)

threeDVar_subsurf([nx, ny, mesh_elements, ...])

Test 3D-Var with 2D Toy Poisson Problem :\(\nabla \cdot (k nabla u) = f\), with:

threeDVar_toy_linear([parameter_size, ...])

Test 3D-Var with 1D Toy Linear Time-independent model :\(b=AX\), with
threeDVar_toy_linear(parameter_size=8, state_size=10, observation_error_variance=0.04, random_seed=1234, observation_operator_configs={'nx': 10, 'type': 'identity'}, prior_variance=0.25, gradient_fd_validation=True, Hessian_fd_validation=True, solve_inverse_problem=True, update_posterior=True, optimization_routine_options={'maxiter': 30}, output_dir='./_PYOED_RESULTS_/Data_Assimilation/3DVAR_LinearToy1D_TimeIndependent', verbose=True)[source]#
Test 3D-Var with 1D Toy Linear Time-independent model :\(b=AX\), with

:\(A\) being a square matrix of size :\((np, nx)\)

Parameters:

  • parameter_size – (:\(np\)) model parameter dimensionality (number of model parameters)

  • state_size – (:\(nx\)) model state dimensionality (model state vector size)

  • random_seed – None, or integer used to control reproduction

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • output_dir – location to save any log or results

threeDVar_subsurf(nx=21, ny=21, mesh_elements='Lagrange', mesh_elements_degree=(1, 2), observation_error_variance=0.0001, random_seed=1234, observation_operator_configs={'nx': 10, 'type': 'identity'}, prior_variance=1.0, prior_type='BiLaplacian', Laplacian_prior_params=(1.0, 0.5), gradient_fd_validation=True, Hessian_fd_validation=False, solve_inverse_problem=True, update_posterior=True, optimization_routine_options={'maxiter': 300}, output_dir='./_PYOED_RESULTS_/Data_Assimilation/3DVAR_Subsurf', verbose=True)[source]#

Test 3D-Var with 2D Toy Poisson Problem :\(\nabla \cdot (k nabla u) = f\), with:

  • :\(k\) being the conductivity field of size :\((nx \times ny)\)

  • :\(u\) being the temperature field of size :\((nx \times ny)\)

  • :\(f\) is the external forcing

Parameters:

  • nx – model dimensionality in the x direction

  • ny – model dimensionality in the y direction

  • mesh_elements (str) – Mesh element type

  • mesh_elements_degree – degree of the mesh elements

  • random_seed – None, or integer used to control reproduction

  • observation_error_variance

  • prior_variance

  • prior_type (str) – name of the prior to use: ‘Gaussian’, ‘Laplacian’, ‘BiLaplacian’

  • Laplacian_prior_params (tuple) – (gamma, delta) passed to Laplaciane-type error models

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • verbose (bool)

  • output_dir – location to save any log or results

main(test)[source]#
Choose the test (4D-var data assimilation for a selected simulation mdoel)

to run and call the proper function from this module

Parameters:

test (str) – name of the test to run; supported names are: - ‘Toy’ : One-dimensional (1D) Linear Toy Model - ‘Poisson’ or ‘Subsurf’: Two-dimensional (2D) Subsurface Model

Raises:

ValueError is raised if the passed test value is not recognized

Four-dimensional variational (4DVar) data assimilation#

Driver script to test the components of 4DVar data assimilation system with several simulation models and observation operators. A user should use these drivers only as examples, and adapt them to their needs.

Functions

main(test)

Choose the test (4D-var data assimilation for a selected simulation mdoel)

fourDVar_AD_1d([domain, nu, c, nx, dt, ...])

Test 4D-Var with 1D Advection-Diffusion model

fourDVar_AD_2d([domain, nu, cx, cy, nx, ny, ...])

Test 4D-Var with 2D Advection-Diffusion model

fourDVar_Brg_1d([domain, nu, nx, dt, ...])

Test 1D Bateman-Burgers

fourDVar_Brg_2d([domain, nu, nx, ny, dt, ...])

Test 2D Bateman-Burgers

fourDVar_Lorenz_63([sigma, rho, beta, ic, ...])

Test Lorenz 63 model

fourDVar_Lorenz_96([nx, F, dt, checkpoints, ...])

Test Lorenz 96 model

fourDVar_Dolfin_AD_2d([dt, checkpoints, ...])

Test 4D-Var with 2D Dolfin-based Advection-Diffusion model
fourDVar_AD_1d(domain=(-1, 1), nu=0.05, c=0.75, nx=51, dt=0.01, checkpoints=array([0., 0.01428571, 0.02857143, 0.04285714, 0.05714286, 0.07142857, 0.08571429, 0.1, 0.11428571, 0.12857143, 0.14285714, 0.15714286, 0.17142857, 0.18571429, 0.2]), observation_error_variance=0.0001, observation_operator_configs={'nx': 15, 'type': 'identity'}, prior_variance=0.01, gradient_fd_validation=True, Hessian_fd_validation=True, solve_inverse_problem=True, update_posterior=True, optimizer_configs={'maxiter': None}, random_seed=1011, output_dir='./_PYOED_RESULTS_/Data_Assimilation/4DVar-AD-1D')[source]#

Test 4D-Var with 1D Advection-Diffusion model

Parameters:

  • domain

  • nu

  • c

  • nx

  • dt

  • checkpoints – [t0, t1, …, tf] observations are made at t1, …, tf

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • random_seed

  • output_dir

fourDVar_AD_2d(domain=((0, 1), (0, 1)), nu=0.01, cx=0.75, cy=0.75, nx=26, ny=26, dt=0.05, checkpoints=array([0., 0.1, 0.2, 0.3, 0.4, 0.5]), observation_operator_configs={'nx': 5, 'ny': 5, 'type': 'identity'}, observation_error_variance=0.0001, prior_variance=0.01, gradient_fd_validation=True, Hessian_fd_validation=True, solve_inverse_problem=True, update_posterior=True, optimizer_configs={'maxiter': None}, random_seed=1011, output_dir='./_PYOED_RESULTS_/Data_Assimilation/4DVar-AD-2D')[source]#

Test 4D-Var with 2D Advection-Diffusion model

Parameters:

  • domain

  • nu

  • c

  • nx

  • dt

  • checkpoints – [t0, t1, …, tf] observations are made at t1, …, tf

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • random_seed

  • output_dir

fourDVar_Brg_1d(domain=(0, 1), nu=0.005, nx=51, dt=0.01, checkpoints=array([0., 0.25, 0.5, 0.75, 1.]), observation_error_variance=0.01, observation_operator_configs={'nx': 15, 'type': 'uniform'}, prior_variance=0.016, gradient_fd_validation=True, Hessian_fd_validation=False, solve_inverse_problem=True, update_posterior=True, optimizer_configs={'maxiter': None}, scr_out_iter=0, random_seed=1011, output_dir='./_PYOED_RESULTS_/Data_Assimilation/4DVar-Bateman-Burgers-1D')[source]#

Test 1D Bateman-Burgers

Parameters:

  • domain

  • nu

  • nx

  • dt

  • checkpoints – [t0, t1, …, tf] observations are made at t1, …, tf

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • random_seed

  • output_dir

fourDVar_Brg_2d(domain=((0, 1), (0, 1)), nu=0.02, nx=11, ny=11, dt=0.01, checkpoints=array([0., 0.1, 0.2, 0.3, 0.4, 0.5]), observation_operator_configs={'nx': 5, 'ny': 5, 'type': 'identity'}, observation_error_variance=0.01, prior_variance=0.1, gradient_fd_validation=True, Hessian_fd_validation=True, solve_inverse_problem=True, update_posterior=True, optimizer_configs={'maxiter': None}, random_seed=1011, output_dir='./_PYOED_RESULTS_/Data_Assimilation/4DVar-Bateman-Burgers-2D')[source]#

Test 2D Bateman-Burgers

Parameters:

  • domain

  • nu

  • nx

  • ny

  • dt

  • checkpoints – [t0, t1, …, tf] observations are made at t1, …, tf

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • output_dir

Pararm random_seed:

fourDVar_Lorenz_63(sigma=10.0, rho=28.0, beta=2.6666666666666665, ic=[1, 1, 1], dt=0.001, checkpoints=array([0., 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1., 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2., 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3., 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 4., 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9, 5., 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 5.9, 6., 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, 7., 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 8., 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, 8.9, 9., 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8, 9.9, 10.]), observation_error_variance=0.01, prior_variance=0.16, gradient_fd_validation=True, Hessian_fd_validation=True, solve_inverse_problem=True, update_posterior=True, optimizer_configs={'maxiter': None}, random_seed=1011, output_dir='./_PYOED_RESULTS_/Data_Assimilation/4DVar-Lorenz-63')[source]#

Test Lorenz 63 model

Parameters:

  • sigma

  • rho

  • beta

  • ic

  • dt

  • checkpoints – [t0, t1, …, tf] observations are made at t1, …, tf

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • output_dir

fourDVar_Lorenz_96(nx=36, F=8.0, dt=0.01, checkpoints=array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.]), observation_error_variance=0.01, prior_variance=0.16, gradient_fd_validation=True, Hessian_fd_validation=True, solve_inverse_problem=True, update_posterior=True, optimizer_configs={'maxiter': None}, random_seed=1011, output_dir='./_PYOED_RESULTS_/Data_Assimilation/4DVar-Lorenz-96')[source]#

Test Lorenz 96 model

Parameters:

  • n

  • F

  • dt

  • checkpoints – [t0, t1, …, tf] observations are made at t1, …, tf

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • random_seed

  • output_dir

fourDVar_Dolfin_AD_2d(dt=0.2, checkpoints=array([0., 0.2, 0.4, 0.6, 0.8, 1.]), observation_operator_configs={'exclude_boxes': [((0.25, 0.15), (0.5, 0.4)), ((0.6, 0.625), (0.75, 0.85))], 'num_obs_points': 25, 'type': 'uniform'}, observation_noise=0.01, prior_variance=1, prior_type='BiLaplacian', Laplacian_prior_params=(1, 16), gradient_fd_validation=True, Hessian_fd_validation=False, solve_inverse_problem=True, update_posterior=True, optimizer_configs={'maxiter': None}, random_seed=1011, output_dir='./_PYOED_RESULTS_/Data_Assimilation/4DVar-Dolfin-AD-2D')[source]#

Test 4D-Var with 2D Dolfin-based Advection-Diffusion model

Parameters:

  • dt

  • checkpoints – [t0, t1, …, tf] observations are made at t1, …, tf

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • random_seed

  • output_dir

main(test)[source]#
Choose the test (4D-var data assimilation for a selected simulation mdoel)

to run and call the proper function from this module

Parameters:

test (str) – name of the test to run; supported names are: - ‘AD1’ : One-dimensional (1D) Advection-Diffusion (AD) model - ‘AD2’ : Two-dimensional (2D) Advection-Diffusion (AD) model - ‘D-AD2’ : Dolfin-based Two-dimensional (2D) Advection-Diffusion (AD) model - ‘Brg1’: One-dimensional (1D) Bateman-Burgers model - ‘Brg2’: Two-dimensional (2D) Bateman-Burgers model - ‘L63’ : Lorenz 63 (three-variable) model - ‘L96’ : Lorenz 96 model

Raises:

ValueError is raised if the passed test value is not recognized

Goal-orienented data assimilation#

Driver script to test the components of a goal-oriented version of 4DVar data assimilation system with fenics-based advection diffusion model

Functions

main()

goal_oriented_fourDVar_Dolfin_AD_2d([dt, ...])

Test 4D-Var with 2D Dolfin-based Advection-Diffusion model
goal_oriented_fourDVar_Dolfin_AD_2d(dt=0.2, checkpoints=array([0., 0.2, 0.4, 0.6, 0.8, 1.]), observation_operator_configs={'exclude_boxes': [((0.25, 0.15), (0.5, 0.4)), ((0.6, 0.625), (0.75, 0.85))], 'num_obs_points': 25, 'type': 'uniform'}, observation_noise=0.01, prior_variance=1, prior_type='BiLaplacian', Laplacian_prior_params=(1, 16), gradient_fd_validation=False, Hessian_fd_validation=False, solve_inverse_problem=False, update_posterior=True, optimization_routine_options={'maxiter': None}, verbose=True, random_seed=1011, output_dir='./_PYOED_RESULTS_/Data_Assimilation/4DVar-Dolfin-AD-2D')[source]#

Test 4D-Var with 2D Dolfin-based Advection-Diffusion model

Parameters:

  • dt

  • checkpoints – [t0, t1, …, tf] observations are made at t1, …, tf

  • observation_error_variance

  • prior_variance

  • gradient_fd_validation (bool) – validate the 4D-Var objective gradient using finite differences

  • random_seed

  • output_dir

main()[source]#