seisflows.system.lsf

This is the subclass seisflows.system.lsf.Lsf

This class provides the core utilities interaction with HPC systems which run using the Platform Load Sharing Facility (LSF) workload management platform.

Module Contents

Classes

Lsf

An interface through which to submit workflows, run tasks in serial or

Functions

_modify_run_call_single_proc(run_call)

class seisflows.system.lsf.Lsf

Bases: seisflows.system.cluster.Cluster

An interface through which to submit workflows, run tasks in serial or parallel, and perform other system functions.

By hiding environment details behind a python interface layer, these classes provide a consistent command set across different computing environments.

Intermediate files are written to a global scratch path self.path.SCRATCH, which must be accessible to all compute nodes.

Optionally, users can provide a local scratch path self.path.LOCAL if each compute node has its own local filesystem.

For important additional information, please see http://seisflows.readthedocs.org/en/latest/manual/

manual.html#system-configuration

submit(submit_call=None)

Submits workflow using ‘bsub’ arguments

run(classname, method, single=False, run_call=None, **kwargs)

Runs task multiple times in embarrassingly parallel fasion on the maui cluster

Executes classname.method(*args, **kwargs) NTASK times, each time on NPROC CPU cores

Parameters

  • classname (str) – the class to run

  • method (str) – the method from the given classname to run

taskid()

Provides a unique identifier for each running task

checkpoint(path, classname, method, kwargs)

Inherits from workflow.system.workstation.Workstation

_check_job_status(job_ids)

Queries completion status of a single job

TODO this function is mangled, needs to be rewritten

Parameters

job (str) – job id to query

_job_id_list(stdout, single)

Parses job id list from LSF standard output

Parameters

  • stdout (str) – the output of subprocess.check_output()

  • single (bool) – if running a single process job, returns a list of length 1 with a single job id, else returns a list of length self.par.NTASK for all arrayed jobs

Return type

list

Returns

a list of array jobs that should be currently running

_query(jobid)

Retrives job state from LSF database

Parameters

jobid (str) – job id to query LSF system about

seisflows.system.lsf._modify_run_call_single_proc(run_call)