seisflows.system.cluster

The Cluster class provides the core utilities interaction with HPC systems which must be overloaded by subclasses for specific workload managers, or specific clusters.

The Cluster class acts as a base class for more specific cluster implementations (like SLURM). However it can be used standalone. When running jobs on the Cluster system, jobs will be submitted to the master system using subprocess.run, mimicing how jobs would be run on a cluster but not actually submitting to any job scheduler.

Module Contents

Classes

Cluster

Cluster System
class seisflows.system.cluster.Cluster(title=None, mpiexec='', ntask_max=None, walltime=10, tasktime=1, environs='', **kwargs)

Bases: seisflows.system.workstation.Workstation

Cluster System

Generic or common HPC/cluster interfacing commands

Parameters

type title

str

param title

The name used to submit jobs to the system, defaults to the name of the current working directory

type mpiexec

str

param mpiexec

Function used to invoke executables on the system. For example ‘mpirun’, ‘mpiexec’, ‘srun’, ‘ibrun’

type ntask_max

int

param ntask_max

limit the number of concurrent tasks in a given array job

type walltime

float

param walltime

maximum job time in minutes for the master SeisFlows job submitted to cluster. Fractions of minutes acceptable.

type tasktime

float

param tasktime

maximum job time in minutes for each job spawned by the SeisFlows master job during a workflow. These include, e.g., running the forward solver, adjoint solver, smoother, kernel combiner. All spawned tasks receive the same task time. Fractions of minutes acceptable.

type environs

str

param environs

Optional environment variables to be provided in the following format VAR1=var1,VAR2=var2… Will be set using os.environs

Paths

***

property submit_call_header

The submit call defines the SBATCH header which is used to submit a workflow task list to the system. It is usually dictated by the system’s required parameters, such as account names and partitions. Submit calls are modified and called by the submit function.

Note

Generalized cluster returns empty string but child system classes will need to overwrite the submit call.

Return type

str

Returns

the system-dependent portion of a submit call

property run_call_header

The run call defines the SBATCH header which is used to run tasks during an executing workflow. Like the submit call its arguments are dictated by the given system. Run calls are modified and called by the run function

Note

Generalized cluster returns empty string but child system classes will need to overwrite the submit call.

Return type

str

Returns

the system-dependent portion of a run call

__doc__
submit(workdir=None, parameter_file='parameters.yaml')

Submits the main workflow job as a separate job submitted directly to the system that is running the master job

Parameters

  • workdir (str) – path to the current working directory

  • parameter_file (str) – paramter file file name used to instantiate the SeisFlows package

run(funcs, single=False, **kwargs)

Runs tasks multiple times in parallel by submitting NTASK new jobs to system. The list of functions and its kwargs are saved as pickles files, and then re-loaded by each submitted process with specific environment variables. Each spawned process will run the list of functions.

Parameters

  • funcs (list of methods) – a list of functions that should be run in order. All kwargs passed to run() will be passed into the functions.

  • single (bool) – run a single-process, non-parallel task, such as smoothing the gradient, which only needs to be run by once. This will change how the job array and the number of tasks is defined, such that the job is submitted as a single-core job to the system.

  • run_call (str) – the call used to submit the run script. If None, attempts default run call which should be suited for the given system. Can be overwritten by child classes to involve other arguments

_run_task(run_call, task_id)

Convenience function to run a single Python job with subprocess.run with some error catching and redirect of stdout to a log file.

Parameters

  • run_call (str) – python call to run a task involving loading the pickled function list and its kwargs, and then running them

  • task_id (int) – given task id, used for log messages and to format the run call