seisflows.optimize.gradient

Gradient descent nonlinear optimization algorithm. Acts as the Base class for optimization.

The Optimization library contains classes and methods used to solve nonlinear optimization problems, i.e., misfit minimization. Various subclasses implement different optimization algorithms.

Note

To reduce memory overhead, and to enable optimization restarts due to failed line searches, model/gradient vectors and line search history are passed on disk rather than in memory.

Note

The default numerical parameters for each algorithm should work well for a range of applications without manual tuning. If the nonlinear optimization procedure stagnates, it may be due to issues involving:

  1. poor data quality

  2. choice of objective function

  3. data processing parameters (i.e., filter bandpass)

  4. regularization methods

Problems in any of these areas usually manifest themselves through stagnation of the nonlinear optimizationalgorithm.

Classes

Gradient

Gradient Optimization [Optimize Base]

Module Contents

class seisflows.optimize.gradient.Gradient(line_search_method='bracket', preconditioner=None, step_count_max=10, step_len_init=0.01, step_len_max=0.1, step_len_min=0.001, workdir=os.getcwd(), path_optimize=None, path_output=None, path_preconditioner=None, **kwargs)

Gradient Optimization [Optimize Base]

Defines foundational structure for Optimization module. Applies a gradient/steepest descent optimization algorithm.

Parameters

type line_search_method:

str

param line_search_method:

chosen line_search algorithm. Currently available are ‘bracket’ and ‘backtrack’. See seisflows.plugins.line_search for all available options

type preconditioner:

str

param preconditioner:

algorithm for preconditioning gradients. Currently available: ‘diagonal’. Requires path_preconditioner to point to a set of files that define the preconditioner, formatted the same as the input model

type step_count_max:

int

param step_count_max:

maximum number of trial steps to perform during the line search before a change in line search behavior is considered, or a line search is considered to have failed.

type step_len_init:

float

param step_len_init:

initial line search step length guess, provided as a fraction of current model parameters.

type step_len_max:

float

param step_len_max:

optional, maximum allowable step length during the line search. Set as a fraction of the current model parameters

type step_len_min:

float

param step_len_min:

optional, minimum allowable step length during the line search. Set as a fraction of the current model parameters

Paths

type path_preconditioner:

str

param path_preconditioner:

optional path to a set of preconditioner files formatted the same as the input model (or output model of solver). Required to exist and contain files if `preconditioner`==True

***

preconditioner = None
step_count_max = 10
step_len_init = 0.01
step_len_max = 0.1
step_len_min = 0.001
path
line_search_method = 'bracket'
_restarted = False
_acceptable_vectors = ['m_new', 'm_old', 'm_try', 'g_new', 'g_old', 'g_try', 'p_new', 'p_old', 'alpha', 'f_new',...
_acceptable_preconditioners = ['diagonal']
__str__()

Quickly access underlying line search search history, mostly for debug purposes

property step_count

Convenience property to access step_count from line search

check()

Checks parameters, paths, and dependencies

setup()

Sets up nonlinear optimization machinery

load_vector(name)

Convenience function to access the full paths of model and gradient vectors that are saved to disk. Corresponding save_vector function saves vectors into the correct location and format that can be loaded by this function.

Note

Model, gradient and misfit vectors are named as follows: m_new: current model m_old: previous model m_try: line search model f_new: current objective function value f_old: previous objective function value f_try: line search function value g_new: current gradient direction g_old: previous gradient direction p_new: current search direction p_old: previous search direction alpha: trial search direction (aka p_try)

Parameters:

name (str) – name of the vector, acceptable: m, g, p, f, alpha

Return type:

Model, vector or float

Returns:

loaded vector corresponding to name that may take on different type based on which vector it is

save_vector(name, m)

Convenience function to save/overwrite vectors on disk. Corresponding function load_vector loads in vectors that have been saved with this function

Parameters:

  • name (str) – name of the vector to save or overwrite

  • m (seisflows.tools.specfem.Model or float) – Model, vector, or value to save to disk

checkpoint()

The optimization module (and its underlying line_search attribute) requires continuity across runs of the same workflow (e.g., in the event of a failed job). This function saves internal attributes of the optimization module to disk so that a resumed workflow does not lose information from its previous version.

User can checkpoint other variables by adding kwargs

load_checkpoint()

Counterpart to optimize.checkpoint. Loads a checkpointed optimization module from disk and sets internal tracking attributes.

_precondition(q)

Apply available preconditioner to a given gradient

Parameters:

q (np.array) – Vector to precondition, typically gradient contained in: g_new

Return type:

np.array

Returns:

preconditioned vector

compute_direction()

Computes steepest descent search direction (inverse gradient) with an optional user-defined preconditioner.

Note

Other optimization algorithms must overload this method

Return type:

seisflows.tools.specfem.Model

Returns:

search direction as a Model instance

Raises:

SystemError – if the search direction is 0, signifying a zero gradient which will not do anything in an update

Setup a the line search machinery by inputting values for the initial model. Also contains a check to see if the line search has been restarted.

Collect information on a forward evaluation that just took place so that we can assess how to proceed with the current line search. Incremenet the line search step count as we have finished the current.

calculate_step_length()

Determine the step length alpha based on the current configuration of the line search machinery (i.e., the misfit vs. function evaluation values). Has a catch for whether we want to manual override the step length. Also returns a status that tells the Workflow how to proceed with a line search.

Rtype alpha:

float

Return alpha:

step length recommended by the line search. This is intrinsically tied to the status. When status`==’TRY’, alpha represents the step length to take for the next step. When `status ==’PASS’, then alpha represents the best fitting step count found for this line search evaluation.

Rtype status:

str

Return status:

the status recommended to the workflow. Three options: Available status returns are: ‘TRY’: try/re-try the line search as conditions have not been met ‘PASS’: line search was successful, you can terminate the search ‘FAIL’: line search has failed for one or more reasons.

compute_trial_model(alpha)

Generates a trial model m_try by perturbing the starting model m_new with a given search direction p_new and a pre-calculated step length alpha

Parameters:

alpha (float) – step length recommended by the line search. if None, due to failed line search, then this function returns None

Return type:

np.array or NoneType

Returns:

trial model that can be used for line search evaluation or None if alpha is None

Prepares algorithm machinery and scratch directory for next model update

Removes old model/search parameters, moves current parameters to old, sets up new current parameters and writes statistic outputs

attempt_line_search_restart(threshold=0.001)

After a failed line search, this determines if restart is worthwhile by checking, in effect, if the search direction was the same as the negative gradient direction.

Essentially checking if this is a steepest-descent optimization, which cannot and should not be restarted. If the search direction is calc’ed by another optimization schema, the search direction and gradient should differ

Parameters:

threshold (float) – angle threshold for the angle between the search direction and the gradient.

Return type:

bool

Returns:

pass (True) fail (False) status for retrying line search

restart()

Restarts nonlinear optimization algorithm for any schema that is NOT steepest descent (default base class).

Keeps current position in model space, but discards history of nonlinear optimization algorithm in an attempt to recover from numerical stagnation.

Note

steepest descent optimization algorithm does not have any restart capabilities. This function is instantiated here to be overwritten by child classes

get_stats()

Get Optimization statistics for the current evaluation of an inversion. Returns a dictionary of values which can then be written to a text file or plotted for convenience.

Note

The following factor was included in the original SeisFlows stats but started throwing runtime errors. Not sure what is was for – BC

factor = -1 * dot(g.vector, g.vector) factor = factor ** -0.5 * (f[1] - f[0]) / (x[1] - x[0])

Return type:

Dict of float

Returns:

SeisFlows dictionary object containing statistical informaion about the optimization module

write_stats(fid='./optim_optim.txt')

Write stats to file so that we don’t lose information to subsequent iterations. File is written to path._output_optim/fid

Parameters:

fid (str) – full path and filename to save the text file. defaults to ./output_optim.txt