seisflows.tools.config

Seisflows configuration tools, containing core utilities that are called upon throughout the Seisflows workflow.

Module Contents

Classes

Dict

A dictionary replacement which allows for easier parameter access through

Null

A null object that always and reliably does nothing

Functions

load_yaml(filename)

Define how the PyYaml yaml loading function behaves.

get_task_id()

Task IDs are assigned to each child process spawned by the system module

set_task_id(task_id)

Set the SEISFLOWS_TASKID in os environs for local workflows. If running

import_seisflows([workdir, parameter_file])

Standard SeisFlows workflow setup block which runs a number of setup

config_logger([level, filename, filemode, verbose, ...])

Explicitely configure the logging module with some parameters defined

custom_import([name, module, classname])

Imports SeisFlows module and extracts class that is the camelcase version

pickle_function_list(functions[, path])

Save a list of functions and their keyword arguments as pickle files.

number_fid(fid[, i])

Number a filename. Used to store old log files without overwriting them.

Attributes

ENV_VARIABLES

seisflows.tools.config.ENV_VARIABLES = ['SEISFLOWS_TASKID', 'SLURM_ARRAY_TASK_ID']
class seisflows.tools.config.Dict

Bases: dict

A dictionary replacement which allows for easier parameter access through getting and setting attributes. Also has some functionality to make string printing prettier

__str__()

Pretty print dictionaries and first level nested dictionaries

__repr__()

Pretty print when calling an instance of this object

__getattr__(key)

Attribute-like access of the internal dictionary attributes

__setattr__(key, val)

Setting attributes can only be performed one time

class seisflows.tools.config.Null(*args, **kwargs)

A null object that always and reliably does nothing

__call__(*args, **kwargs)
__bool__()
__nonzero__()
__getattr__(key)
__setattr__(key, val)

Implement setattr(self, name, value).

__delattr__(key)

Implement delattr(self, name).

seisflows.tools.config.load_yaml(filename)

Define how the PyYaml yaml loading function behaves. Replaces None and inf strings with NoneType and numpy.inf respectively Also expands all paths (parameters that start with ‘path_’) to be absolute

Parameters

filename (str) – .yaml file to load in

Return type

Dict

Returns

Dictionary containing all parameters in a YAML file

seisflows.tools.config.get_task_id()

Task IDs are assigned to each child process spawned by the system module during a SeisFlows workflow. SeisFlows modules use this Task ID to keep track of embarassingly parallel process, e.g., solver uses the Task ID to determine which source is being considered.

Return type

int

Returns

task id for given solver

seisflows.tools.config.set_task_id(task_id)

Set the SEISFLOWS_TASKID in os environs for local workflows. If running on HPC systems, running array jobs will assign the Task ID

Note

Mostly used for debugging/testing purposes as a way of mimicing system.run() assigning task ids to child processes

Parameters

task_id (int) – integer task id to assign to the current working environment

seisflows.tools.config.import_seisflows(workdir=os.getcwd(), parameter_file='parameters.yaml', **kwargs)

Standard SeisFlows workflow setup block which runs a number of setup tasks including: loading a user-defined parameter file, configuring the package-wide logger based on user-input path to log file and desired verbosity, and instantiating all modules in a generic fashion based on user choice. Returns the ‘workflow’ module, which contains all other submodules as attributes.

Parameters

  • workdir (str) – the current working directory in which to perform a SeisFlows workflow. Defaults to the current working directory

  • parameter_file (str) – the YAML formatted parameter file that is used to instantiate each of the SeisFlows modules and run the workflow. This should be created by the command line argument ‘seisflows configure’. Defaults to ‘parameters.yaml’

Return type

module

Returns

instantiated ‘workflow’ module which contains all sub-modules which have been instantiated with user-defined parameters

seisflows.tools.config.config_logger(level='DEBUG', filename=None, filemode='a', verbose=True, stream_handler=True)

Explicitely configure the logging module with some parameters defined by the user in the System module. Instantiates a stream logger to write to stdout, and a file logger which writes to filename. Two levels of verbosity and three levels of log messages allow the user to determine how much output they want to see.

Parameters

  • level (str) – log level to be passed to logger, available are ‘CRITICAL’, ‘WARNING’, ‘INFO’, ‘DEBUG’

  • filename (str or None) – name of the log file to write log statements to. If None, logs will be written to STDOUT ONLY, and filemode will not be used.

  • filemode (str) – method for opening the log file. defaults to append ‘a’

  • verbose (bool) – if True, writes a more detailed log message stating the type of log (warning, info, debug), and the class and method which called the logger (e.g., seisflows.solver.specfem2d.save()). This is much more useful for debugging but clutters up the log file. if False, only write the time and message in the log statement.

seisflows.tools.config.custom_import(name=None, module=None, classname=None)

Imports SeisFlows module and extracts class that is the camelcase version of the module name. Used to dynamically import sub-modules by name only, avoiding the need to hardcode import statements.

For example:

custom_import(‘workflow’, ‘inversion’)

imports ‘seisflows.workflow.inversion’ and, from this module, extracts class ‘Inversion’.

Parameters

  • name (str) –

    component of the workflow to import, defined by names, available: “system”, “preprocess”, “solver”,

    ”postprocess”, “optimize”, “workflow”

  • classname (str) – the class to be called from the module. Usually this is just the CamelCase version of the module, which will be defaulted to if this parameter is set None, however allows for custom class naming. Note: CamelCase class names following PEP-8 convention.

seisflows.tools.config.pickle_function_list(functions, path=os.getcwd(), **kwargs)

Save a list of functions and their keyword arguments as pickle files. Return the names of the files. Used for running functions from spawned processes during cluster runs.

Note

The idea here is that we need this list of functions to be discoverable by a system separate to the one that defined them. To do this we can pickle Python objects on disk, and have the new system read in the pickle files and evaluate the objects. We use ‘dill’ because Pickle can’t serialize methods/functions

Parameters

  • functions (list of methods) – a list of functions that should be run in order. All kwargs passed to run() will be passed into the functions.

  • path (str) – path to save the pickle files. Defaults to current working directory

Return type

tuple of str

Returns

(name of the pickle file containing the function, name of the pickle file containing keyword arguments)

seisflows.tools.config.number_fid(fid, i=0)

Number a filename. Used to store old log files without overwriting them. Premise is, if you have a file e.g., called: output.txt This function would return incrementing filenames: output_000.txt, output_001.txt, output_002.txt, ouput_003.txt …

Note

Replace statement is catch-all, so we assume that there is only one instance of the file extension in the entire path.

Parameters

  • fid (str) – path to the file that you want to increment

  • i (int) – number to append to file id

Return type

str

Returns

filename with appended number. filename ONLY, will strip away the original path location