seisflows.system.wisteria

Wisteria is the University of Tokyo Fujitsu brand high performance computer. Wisteria runs on the Fujitsu/PJM job scheduler.

Note

  • Wisteria has two node groups, Odyssey (compute nodes) and Aquarius (data/learning nodes w/ GPU)

  • Odyssey has 7680 nodes with 48 cores/node

  • Aquarius has 45 nodes with 36 cores/node

  • Aquarius also contains 8x Nvidia A100

Note

Wisteria Caveat 1

On Wisteria you cannot submit batch jobs from compute nodes and you cannot SSH from compute nodes (Manual 5.13), so the master job must be run from the login node or the pre-post node (Manual 5.2.3)

Note

Wisteria Caveat 2

On Wisteria, the login node Conda environment is not inherited by compute nodes, so it requires custom submit and run script which first load the correct modules, and then run the corresponding script

Note

Wisteria Caveat 3

On Wisteria, command line arguments for the submit and run script, normally input like ‘–key value’ interfere with the batch submission cmd pjsub. So instead we use the pjsub ‘-x’ flag which allows us to set environment variables. We use these in place of command line arguments

Classes

Wisteria

System Wisteria

Module Contents

class seisflows.system.wisteria.Wisteria(group=None, rscgrp=None, gpu=None, **kwargs)

Bases: seisflows.system.fujitsu.Fujitsu

System Wisteria

University of Tokyo HPC Wisteria, running Fujitsu job scheduler

Parameters

type group:

str

param group:

User’s group for allocating and charging resources. In the pjsub script this is the ‘-g’ option.

type rscgrp:

str

param rscgrp:

the resource group (i.e., partition) to submit jobs to. In the pjsub script this is the ‘-L rscgrp’ option. Available `rscgrp`s for Wisteria are:

  • debug-o: Odyssey debug, 30 min max, [1, 144] nodes available

  • short-o: Odyssey short, 8 hr. max, [1, 72] nodes available

  • regular-o: Odyssey regular, 24-48 hr. max, [1, 2304] nodes available

  • priority-o: Odyssey priority, 48 hr. max, [1, 288] nodes available

  • debug-a: Aquarius debug, 30 min max, [1, 1] nodes available

  • short-a: Aquarius short, 2 hr. max, [1, 2] nodes available

  • regular-a: Aquarius regular, 24-48 hr. max, [1, 8] nodes available

  • share-debug: Aquarius GPU debug, 30 min max, 1, 2, 4 GPU available

  • share-short: Aquarius GPU short queue, 2 hr. max, 1, 2, 4 GPU avail.

type gpu:

int

param gpu:

if not None, tells SeisFlows to use the GPU version of SPECFEM, the integer value of gpu will set the number of requested GPUs for a simulation on system (i.e., #PJM -L gpu=`gpu`)

Paths

***

__doc__
submit_workflow
run_functions
group = None
rscgrp = None
gpu = None
_rscgrps