seisflows.optimize.gradient
Gradient descent nonlinear optimization algorithm. Acts as the Base class for optimization.
The Optimization library contains classes and methods used to solve nonlinear optimization problems, i.e., misfit minimization. Various subclasses implement different optimization algorithms.
Note
To reduce memory overhead, and to enable optimization restarts due to failed line searches, model/gradient vectors and line search history are passed on disk rather than in memory.
Note
The default numerical parameters for each algorithm should work well for a range of applications without manual tuning. If the nonlinear optimization procedure stagnates, it may be due to issues involving:
poor data quality
choice of objective function
data processing parameters (i.e., filter bandpass)
regularization methods
Problems in any of these areas usually manifest themselves through stagnation of the nonlinear optimizationalgorithm.
Module Contents
Classes
Gradient Optimization |
- class seisflows.optimize.gradient.Gradient(line_search_method='bracket', preconditioner=None, step_count_max=10, step_len_init=0.05, step_len_max=0.5, workdir=os.getcwd(), path_optimize=None, path_output=None, path_preconditioner=None, **kwargs)
Gradient Optimization
Defines foundational structure for Optimization module. Applies a gradient/steepest descent optimization algorithm.
Parameters
- type line_search_method
str
- param line_search_method
chosen line_search algorithm. Currently available are ‘bracket’ and ‘backtrack’. See seisflows.plugins.line_search for all available options
- type preconditioner
str
- param preconditioner
algorithm for preconditioning gradients. Currently available: ‘diagonal’. Requires path_preconditioner to point to a set of files that define the preconditioner, formatted the same as the input model
- type step_count_max
int
- param step_count_max
maximum number of trial steps to perform during the line search before a change in line search behavior is considered, or a line search is considered to have failed.
- type step_len_init
float
- param step_len_init
initial line search step length guess, provided as a fraction of current model parameters.
- type step_len_max
float
- param step_len_max
maximum allowable step length during the line search. Set as a fraction of the current model parameters
Paths
- type path_preconditioner
str
- param path_preconditioner
optional path to a set of preconditioner files formatted the same as the input model (or output model of solver). Required to exist and contain files if `preconditioner`==True
- property step_count
Convenience property to access step_count from line search
- check()
Checks parameters, paths, and dependencies
- setup()
Sets up nonlinear optimization machinery
- load_vector(name)
Convenience function to access the full paths of model and gradient vectors that are saved to disk
Note
Model, gradient and misfit vectors are named as follows: m_new: current model m_old: previous model m_try: line search model f_new: current objective function value f_old: previous objective function value f_try: line search function value g_new: current gradient direction g_old: previous gradient direction p_new: current search direction p_old: previous search direction alpha: trial search direction (aka p_try)
- Parameters
name (str) – name of the vector, acceptable: m, g, p, f, alpha
- save_vector(name, m)
Convenience function to save/overwrite vectors on disk
- Parameters
name (str) – name of the vector to overwrite
m (seisflows.tools.specfem.Model or float) – Model to save to disk as npz array
- checkpoint()
The optimization module (and its underlying line_search attribute) requires continuity across runs of the same workflow (e.g., in the event of a failed job). This function saves internal attributes of the optimization module to disk so that a resumed workflow does not lose information from its previous version.
User can checkpoint other variables by adding kwargs
- load_checkpoint()
Counterpart to optimize.checkpoint. Loads a checkpointed optimization module from disk and sets internal tracking attributes.
- _precondition(q)
Apply available preconditioner to a given gradient
- Parameters
q (np.array) – Vector to precondition, typically gradient contained in: g_new
- Return type
np.array
- Returns
preconditioned vector
- compute_direction()
Computes steepest descent search direction (inverse gradient) with an optional user-defined preconditioner.
Note
Other optimization algorithms must overload this method
- Return type
seisflows.tools.specfem.Model
- Returns
search direction as a Model instance
- initialize_search()
Generate a trial model by perturbing the current model in the search direction with a given step length, calculated by the chosen line search algorithm.
- Return type
tuple
- Returns
(Model, float) or (m_try==trial model, alpha=step length)
- update_line_search()
Updates line search status and step length after a forward simulation has been run and misfit calculated. Checks misfit against line search history to see if the line search has been completed.
Note
This is a bit confusing as it calculates the step length alpha for the NEXT line search step, while storing the alpha value that was calculated from the LAST line search step. This is because we need a corresponding misfit f_try from the value of alpha, which happens externally with the solver module
If line search returns a passing exit code (0 or 1), sets up for a subsequent line search evaluation by saving a new step length (alpha), and creating a new trial model (m_try).
- Return type
tuple
- Returns
(Model, float, bool) or (m_try==trial model, alpha=step length, status==how to proceed with line search)
- finalize_search()
Prepares algorithm machinery and scratch directory for next model update
Removes old model/search parameters, moves current parameters to old, sets up new current parameters and writes statistic outputs
- attempt_line_search_restart(threshold=0.001)
After a failed line search, this determines if restart is worthwhile by checking, in effect, if the search direction was the same as the negative gradientdirection.
Essentially checking if this is a steepest-descent optimization, which cannot and should not be restarted. If the search direction is calc’ed by another optimization schema, the search direction and gradient should differ
- Parameters
threshold (float) – angle threshold for the angle between the search direction and the gradient.
- Return type
bool
- Returns
pass (True) fail (False) status for retrying line search
- restart()
Restarts nonlinear optimization algorithm for any schema that is NOT steepest descent (default base class).
Keeps current position in model space, but discards history of nonlinear optimization algorithm in an attempt to recover from numerical stagnation.
Note
steepest descent optimization algorithm does not have any restart capabilities. This function is instantiated here to be overwritten by child classes
- _write_stats()
Simplified write function to append values to text files. Used because stats line search information can be overwritten by subsequent iterations so we need to append values to text files if they should be retained.
Note
This CSV file can be easily read and plotted using np.genfromtxt >>> np.genfromtxt(“optim_stats.txt”, delimiter=”,”, names=True, dtype=None)