seisflows.solver.specfem

This Solver module is in charge of interacting with external numerical solvers such as SPECFEM (2D/3D/3D_GLOBE). This SPECFEM base class provides general functions that work with all versions of SPECFEM. Subclasses will provide additional capabilities unique to each version of SPECFEM.

Note

The Base class implementation is almost completely SPECFEM2D related. However, SPECFEM2D requires a few unique parameters that 3D/3D_GLOBE do not. Because of the inheritance architecture of SeisFlows, we do not want the 3D and 3D_GLOBE versions to inherit 2D-specific parameters, so we need this this more generalized SPECFEM base class.

TODO

• add in apply_hess functionality that was partially written in legacy code

• move _initialize_adjoint_traces to workflow.migration

• Add density scaling based on Vp?

Module Contents

Classes

Specfem

Solver SPECFEM

class seisflows.solver.specfem.Specfem(syn_data_format='ascii', materials='acoustic', density=False, nproc=1, ntask=1, attenuation=False, smooth_h=0.0, smooth_v=0.0, components='ZNE', source_prefix=None, mpiexec=None, workdir=os.getcwd(), path_solver=None, path_eval_grad=None, path_data=None, path_specfem_bin=None, path_specfem_data=None, path_model_init=None, path_model_true=None, path_output=None, **kwargs)

Solver SPECFEM

Defines foundational structure for Specfem-based solver module. Generalized SPECFEM interface to manipulate SPECFEM2D/3D/3D_GLOBE w/ Python

Parameters

type syn_data_format

str

param syn_data_format

data format for reading synthetic traces into memory. Available: [‘SU’: seismic unix format, ‘ASCII’: human-readable ascii]

type materials

str

param materials

Material parameters used to define model. Available: [‘ELASTIC’: Vp, Vs, ‘ACOUSTIC’: Vp, ‘ISOTROPIC’, ‘ANISOTROPIC’]

type density

bool

param density

How to treat density during inversion. If True, updates density during inversion. If False, keeps it constant. TODO allow density scaling during an inversion

type attenuation

bool

param attenuation

How to treat attenuation during inversion. if True, turns on attenuation during forward simulations only. If False, attenuation is always set to False. Requires underlying attenution (Q_mu, Q_kappa) model

type smooth_h

float

param smooth_h

Gaussian half-width for horizontal smoothing in units of meters. If 0., no smoothing applied. Only applicable for workflows: [‘migration’, ‘inversion’], ignored for ‘forward’ workflow. SPECFEM3D_GLOBE only: if `smooth_type`==’laplacian’ then this is just the X and Y extent of the applied smoothing

type smooth_h

float

param smooth_v

Gaussian half-width for vertical smoothing in units of meters. Only applicable for workflows: [‘migration’, ‘inversion’], ignored for ‘forward’ workflow. SPECFEM3D_GLOBE only: if `smooth_type`==’laplacian’ then this is just the Z extent of the applied smoothing

type components

str

param components

components to consider and tag data with. Should be string of letters such as ‘RTZ’

type solver_io

str

param solver_io

format of model/kernel/gradient files expected by the numerical solver. Available: [‘fortran_binary’: default .bin files]. TODO: [‘adios’: ADIOS formatted files]

type source_prefix

str

param source_prefix

prefix of source/event/earthquake files. If None, will attempt to guess based on the specific solver chosen.

type mpiexec

str

param mpiexec

MPI executable used to run parallel processes. Should also be defined for the system module

Paths

type path_data

str

param path_data

path to any externally stored data required by the solver

type path_specfem_bin

str

param path_specfem_bin

path to SPECFEM bin/ directory which contains binary executables for running SPECFEM

type path_specfem_data

str

param path_specfem_data

path to SPECFEM DATA/ directory which must contain the CMTSOLUTION, STATIONS and Par_file files used for running SPECFEM

***

property source_names

Returns list of source names which should be stored in PAR.SPECFEM_DATA Source names are expected to match the following wildcard, ‘PREFIX_*’ where PREFIX is something like ‘CMTSOLUTION’ or ‘FORCE’

Note

Dependent on environment variable ‘SEISFLOWS_TASKID’ which is assigned by system.run() to each individually running process.

Return type

list

Returns

list of source names

property source_name

Returns name of source currently under consideration

Note

Dependent on environment variable ‘SEISFLOWS_TASKID’ which is assigned by system.run() to each individually running process.

Return type

str

Returns

given source name for given task id

property cwd

Returns working directory currently in use by a running solver instance

Note

Dependent on environment variable ‘SEISFLOWS_TASKID’ which is assigned by system.run() to each individually running process.

Return type

str

Returns

current solver working directory

property model_databases

The location of model inputs and outputs as defined by SPECFEM2D. This is RELATIVE to a SPECFEM2D working directory.

Note

This path is SPECFEM version dependent so SPECFEM3D/3D_GLOBE versions must overwrite this function

Return type

str

Returns

path where SPECFEM2D database files are stored, relative to solver.cwd

property model_files

Return a list of paths to model files that match the internal parameter list. Used to generate model vectors of the same length as gradients.

Return type

list

Returns

a list of full paths to model files that matches the internal list of solver parameters

property kernel_databases

The location of kernel inputs and outputs as defined by SPECFEM2D This is RELATIVE to a SPECFEM2D working directory.

Note

This path is SPECFEM version dependent so SPECFEM3D/3D_GLOBE versions must overwrite this function

Return type

str

Returns

path where SPECFEM2D database files are stored, relative to solver.cwd

check()

Checks parameter validity for SPECFEM input files and model parameters

data_wildcard(comp='?')

Returns a wildcard identifier for synthetic data based on SPECFEM2D file naming schema. Allows formatting dcomponent e.g., when called by solver.data_filenames.

Note

SPECFEM3D/3D_GLOBE versions must overwrite this function

Parameters

comp (str) – component formatter, defaults to wildcard ‘?’

Return type

str

Returns

wildcard identifier for channels

model_wildcard(par='*', kernel=False)

Returns a wildcard identifier to search for models kernels generated by the solver. An example SPECFEM2D/3D kernel filename (in FORTRAN binary file format) is: ‘proc000001_rho_kernel.bin’ Whereas the corresponding model would be ‘proc000001_rho.bin’

Allows dynamically searching for specific files when renaming, moving or copying files. Also allows for different wildcard for 3D_GLOBE version

Parameters

• comp (str) – component formatter, defaults to wildcard ‘?’

• kernel (bool) – wildcarding a kernel file. If True, adds the ‘kernel’ tag. If not, assuming we are wildcarding for a model file

Return type

str

Returns

wildcard identifier for channels

data_filenames(choice='obs')

Returns the filenames of SPECFEM2D data, either by the requested components or by all available files in the directory.

Note

SPECFEM3D/3D_GLOBE versions must overwrite this function

Note

If the glob returns an empty list, this function exits the workflow because filenames should not be empty is they’re being queried

Return type

list

Returns

list of data filenames

setup()

Prepares solver scratch directories for an impending workflow.

Sets up directory structure expected by SPECFEM and copies or generates seismic data to be inverted or migrated.

Exports INIT/STARTING and TRUE/TARGET models to disk (output/ dir.)

forward_simulation(executables=None, save_traces=False, export_traces=False, **kwargs)

Wrapper for SPECFEM binaries: ‘xmeshfem?D’ ‘xgenerate_databases’,

‘xspecfem?D’

Calls SPECFEM2D forward solver, exports solver outputs to traces dir

Note

SPECFEM3D/3D_GLOBE versions must overwrite this function

Parameters

• executables (list or None) – list of SPECFEM executables to run, in order, to complete a forward simulation. This can be left None in most cases, which will select default values based on the specific solver being called (2D/3D/3D_GLOBE). It is made an optional parameter to keep the function more general for inheritance purposes.

• save_traces (str) – move files from their native SPECFEM output location to another directory. This is used to move output waveforms to ‘traces/obs’ or ‘traces/syn’ so that SeisFlows knows where to look for them, and so that SPECFEM doesn’t overwrite existing files during subsequent forward simulations

• export_traces (str) – export traces from the scratch directory to a more permanent storage location. i.e., copy files from their original location

adjoint_simulation(executables=None, save_kernels=False, export_kernels=False)

Wrapper for SPECFEM binary ‘xspecfem?D’

Calls SPECFEM2D adjoint solver, creates the SEM folder with adjoint traces which is required by the adjoint solver. Renames kernels after they have been created from ‘alpha’ and ‘beta’ to ‘vp’ and ‘vs’, respectively.

Note

SPECFEM3D/3D_GLOBE versions must overwrite this function

Parameters

• executables (list or None) – list of SPECFEM executables to run, in order, to complete an adjoint simulation. This can be left None in most cases, which will select default values based on the specific solver being called (2D/3D/3D_GLOBE). It is made an optional parameter to keep the function more general for inheritance purposes.

• save_kernels (str) – move the kernels from their native SPECFEM output location to another path. This is used to move kernels to another SeisFlows scratch directory so that they are discoverable by other modules. The typical location they are moved to is path_eval_grad

• export_kernels (str) – export/copy/save kernels from the scratch directory to a more permanent storage location. i.e., copy files from their original location. Note that kernel file sizes are LARGE, so exporting kernels can lead to massive storage requirements.

combine(input_path, output_path, parameters=None)

Wrapper for ‘xcombine_sem’. Sums kernels from individual source contributions to create gradient.

Note

The binary xcombine_sem simply sums matching databases

Note

It is ASSUMED that this function is being called by system.run(single=True) so that we can use the main solver directory to perform the kernel summation task

Parameters

• input_path (str) – path to data

• output_path (strs) – path to export the outputs of xcombine_sem

• parameters (list) – optional list of parameters, defaults to self._parameters

smooth(input_path, output_path, parameters=None, span_h=None, span_v=None, use_gpu=False)

Wrapper for SPECFEM binary: xsmooth_sem Smooths kernels by convolving them with a 3D Gaussian

Note

It is ASSUMED that this function is being called by system.run(single=True) so that we can use the main solver directory to perform the kernel smooth task

Parameters

• input_path (str) – path to data

• output_path (str) – path to export the outputs of xcombine_sem

• parameters (list) – optional list of parameters, defaults to self._parameters

• span_h (float) – horizontal smoothing length in meters

• span_v (float) – vertical smoothing length in meters

• use_gpu (bool) – whether to use GPU acceleration for smoothing. Requires GPU compiled binaries and GPU compute node.

_run_binary(executable, stdout='solver.log', with_mpi=True)

Calls MPI solver executable to run solver binaries, used by individual processes to run the solver on system. If the external solver returns a non-zero exit code (failure), this function will return a negative boolean.

Note

This function ASSUMES it is being run from a SPECFEM working directory, i.e., that the executables are located in ./bin/

Note

This is essentially an error-catching wrapper of subprocess.run()

Parameters

• executable (str) – executable function to call. May or may not start E.g., acceptable calls for the solver would ‘./bin/xspecfem2D’. Also accepts additional command line arguments such as: ‘xcombine_sem alpha_kernel kernel_paths…’

• stdout (str) – where to redirect stdout

• with_mpi (bool) – If mpiexec is given, use MPI to run the executable. Some executables (e.g., combine_vol_data_vtk) must be run in serial so this flag allows them to turn off MPI running.

Raises

SystemExit – If external numerical solver return any failure code while running

static _exc2log(exc)

Very simple conversion utility to get log file names based on binaries. e.g., binary ‘xspecfem2D’ will return ‘solver’. Helps keep log file naming consistent and generalizable

TODO add a check here to see if the log file exists, and then use

number_fid to increment so that we keep all the output logs

Parameters

exc (str) – specfem executable, e.g., xspecfem2D, xgenerate_databases

Return type

str

Returns

logfile name that matches executable name

import_model(path_model)

Copy files from given path_model into the current working directory model database. Used for grabbing starting models (e.g., MODEL_INIT) and models that have been perturbed by the optimization library.

Parameters

path_model (str) – path to an existing starting model

_initialize_working_directories()

Serial task used to initialize working directories for each of the a available sources

_initialize_working_directory(cwd=None)

Creates scratch directory structure expected by SPECFEM (i.e., bin, DATA, OUTPUT_FILES). Copies executables (bin) and input data (DATA) directories, prepares simulation input files.

Each directory will act as completely independent Specfem working dir. This allows for embarrassing parallelization while avoiding the need for intra-directory communications, at the cost of temporary disk space.

Note

path to binary executables must be supplied by user as SeisFlows has no mechanism for automatically compiling from source code.

Parameters

cwd (str) – optional scratch working directory to intialize. If None, will set based on current running seisflows task (self.taskid)

_export_starting_models(parameters=None)

Export the initial and target models to the SeisFlows output/ directory.

Parameters

parameters (list) – list of parameters to export. If None, will default to self._parameters