config.yaml options¶

The different running modes of benchcab are solely dependent on the options used in config.yaml. The following gives some typical ways to configure benchcab for each mode, but the tool is not restricted to these choices of options:

Regression testNew feature testEnsemble mode

For this test, you want to:

Specify the details of two branches of CABLE
Do not specify a patch
Use the default set of science options, i.e. do not specify science_configurations in config.yaml
Choose the experiment suitable for your stage of development. A run with the forty-two-site-test will be required for submissions of new development to CABLE.

For this test, you want to:

Specify the details of two branches of CABLE
Specify a patch for one of the branches
Use the default set of science options, i.e. do not specify science_configurations in config.yaml
Choose the experiment suitable for your stage of development. A run with the forty-two-site-test will be required for submissions of new development to CABLE.

This running mode is quite open to customisations:

Specify the number of CABLE's branches you need
Use patch on branches as required
Specify the science configurations you want to run. patch will be applied on top of the science configurations listed.

project¶

Default: user's default project, optional key. NCI project ID to charge the simulations to. The user's default project defined in the $PROJECT environment variable is used by default.

project: nf33

modules¶

Default: required key, no default. NCI modules to use for compiling CABLE

modules: [
  intel-compiler/2021.1.1,
  netcdf/4.7.4,
  openmpi/4.1.0
]

fluxsite¶

Contains settings specific to fluxsite tests.

This key is optional. Default settings for the fluxsite tests will be used if it is not present

fluxsite:
  experiment: AU-How
  pbs:
    ncpus: 18
    mem: 30GB
    walltime: 06:00:00
    storage: [scratch/a00, gdata/xy11]
  multiprocess: True

experiment ¶

Default: forty-two-site-test, optional key. Type of fluxsite experiment to run. Experiments are defined in the benchcab-evaluation workspace on modelevaluation.org. This key specifies the met forcing files used in the test suite. To choose from:

Key value	Experiment description
`forty-two-site-test`	Run simulations for 42 FLUXNET sites
`five-site-test`	Run simulations for 5 FLUXNET sites
`AU-Tum`	Run simulations at the Tumbarumba (AU) site
`AU-How`	Run simulations at the Howard Spring (AU) site
`FI-Hyy`	Run simulations at the Hyytiala (FI) site
`US-Var`	Run simulations at the Vaira Ranch-Ione (US) site
`US-Whs`	Run simulations at the Walnut Gulch Lucky Hills Shrub (US) site

fluxsite:
  experiment: AU-How

pbs ¶

Contains settings specific to the PBS scheduler at NCI for the PBS script running the CABLE simulations at FLUXNET sites and the bitwise comparison for these simulations.

This key is optional. Default values for the PBS settings will apply if it is not specified.

fluxsite:
  pbs:
    ncpus: 18
    mem: 30GB
    walltime: 06:00:00
    storage: [scratch/a00, gdata/xy11]

ncpus: Default: 18, optional key. The number of CPU cores to allocate for the PBS job, i.e. the -l ncpus=<4> PBS flag in PBS Directives Explained.

fluxsite:
  pbs:
    ncpus: 18

mem: Default: 30GB, optional key. The total memory limit for the PBS job, i.e. the -l mem=<10GB> PBS flag in PBS Directives Explained.

fluxsite:
  pbs:
    mem: 30GB

walltime: Default: 6:00:00, optional key. The wall clock time limit for the PBS job, i.e. -l walltime=<HH:MM:SS> PBS flag in PBS Directives Explained.
fluxsite: pbs: walltime: 6:00:00
storage: Default: [], optional key. List of extra storage flags required for the PBS job, i.e. -l storage=<scratch/a00> in PBS Directives Explained.

fluxsite:
  pbs:
    storage: [scratch/a00, gdata/xy11]

multiprocess ¶

Default: True, optional key. Enables or disables multiprocessing for executing embarrassingly parallel tasks.

fluxsites:
  multiprocess: True

spatial¶

Contains settings specific to spatial tests.

This key is optional. Default settings for the spatial tests will be used if it is not present.

spatial:
  met_forcings:
    crujra_access: https://github.com/CABLE-LSM/cable_example.git
  payu:
    config:
      walltime: 1:00:00
    args: -n 2

met_forcings ¶

Specify one or more spatial met forcings to use in the spatial test suite. Each entry is a key-value pair where the key is the name of the met forcing and the value is a URL to a payu experiment that is configured to run CABLE with that forcing.

This key is optional. Default values for the met_forcings key is as follows:

spatial:
  met_forcings:
    crujra_access: https://github.com/CABLE-LSM/cable_example.git

payu ¶

Contains settings specific to the payu workflow manager.

This key is optional. Default values for the payu settings will apply if not specified.

spatial:
  payu:
    config:
      walltime: 1:00:00
    args: -n 2

config: Default: unset, optional key. Specify global configuration options for running payu. Settings specified here are passed into to the payu configuration file for each experiment.

spatial:
  payu:
    config:
      walltime: 1:00:00

args: Default: unset, optional key. Specify command line arguments to the payu run command in the form of a string. Arguments are used for all spatial payu runs.

spatial:
  payu:
    args: -n 2

realisations¶

Entries for each CABLE branch to use. Each entry is a key-value pair and are listed as follows:

realisations:
  # head of main branch
  - repo:
      git:
        branch: main
  # some development branch
  - repo:
      git:
        branch: my_branch
    patch:
      cable:
        cable_user:
          FWSOIL_SWITCH: "Lai and Ktaul 2000"
    patch_remove:
      cable:
        soilparmnew: nil

repo ¶

Contains settings to specify the CABLE branch to test against.

This key is required. The repo key must specify the svn, the git or the local key.

realisations:
  - repo:
      svn:
        branch_path: trunk
  - repo:
      git:
        branch: main
  - repo:
      local:
        path: /home/ab1234/cable_local

`svn`¶

Contains settings to specify a branch from the CABLE SVN repository (https://trac.nci.org.au/svn/cable).

This key is optional. No default.

realisations:
  - repo:
      svn:
        branch_path: branches/Users/foo/my_branch
        revision: 1234

branch_path: Default: required key, no default. Specify the branch path relative to the SVN root of the CABLE repository (https://trac.nci.org.au/svn/cable).

realisations:
  # head of the trunk
  - repo:
      svn:
        branch_path: trunk # (1)
  # some development branch
  - repo:
      svn:
        branch_path: branches/Users/foo/my_branch # (2)

To checkout https://trac.nci.org.au/svn/cable/trunk
To checkout https://trac.nci.org.au/svn/cable/branches/Users/foo/my_branch

revision: Default: HEAD of the branch is checked out, optional key. Specify the revision number to checkout for the branch. This option can be used to ensure the reproducibility of the tests.

realisations:
  - repo:
      svn:
        branch_path: branches/Users/foo/my_branch
        revision: 1234

`git`¶

Contains settings to specify a branch on the GitHub repository. By default, the CABLE GitHub repository will be cloned (see url to specify another GitHub repository).

This key is optional. No default.

realisations:
  - repo:
      git:
        branch: my_branch
        commit: 067b1f4a570385fce01552fdf96ced0adbbe17eb
        url: https://github.com/SeanBryan51/CABLE.git

branch: Default: required key, no default. Specify the GitHub branch name to be checked out.

realisations:
  - repo:
      git:
        branch: my_branch

commit: Default: unset, optional key. Specify a specific commit to use for the branch.

realisations:
  - repo:
      git:
        branch: my_branch
        commit: 067b1f4a570385fce01552fdf96ced0adbbe17eb

url: Default: URL of the CABLE GitHub repository, optional key. Specify the GitHub repository url to clone from when checking out the branch.

`local`¶

Contains settings to specify CABLE checkouts on a local repository.

This key is optional. No default.

realisations:
  - repo:
      local:
        path: /scratch/tm70/ab1234/CABLE

path: Default: required key, no default. Specify the local checkout path of CABLE branch.

meorg_output_name ¶

Default: unset, optional key. Chosen as the model name for one of the realisations, if the user wants to upload the Model Output to me.org for further analysis. The following workflow is executed:

A model_output_name is created using the format <realisation_name>-<hash>. Here, The realisation_name is determined where meorg_output_name is set as true.
Note: The realisation_name is set via name if provided, otherwise the default repository name is used. A 6-character hash derived from realisations, model_profile_id and $USER is appended at the end. The hash is used to minimise name conflicts for different users' needs.
Note: In case model_output_name already exists on me.org, the files within that model output are deleted. This is done to send a fresh set of benchmarking results for analysis, ensuring that the user can re-run benchcab without any issues.
The following settings are taken by default for the model output:
- Model Profile - CABLE
- State Selection - default
- Parameter Selection - automated
- Bundled experiments - true
- Comments - none
Depending on the fluxsite experiment, benchcab will do the following:
Add the correponding experiment to model output.
Associate the experiment with base benchmark (already stored in me.org), and other listed realisations (since they share the same experiment).
Run the analysis, and provide a link to the user to check status.

The model output name should also follow the Github issue branch format (i.e. it should start with a digit, with words separated by dashes). Finally, the maximum number of characters allowed for meorg_output_name is 50.

This key is optional. No default.

realisations:
  - repo:
      git:
        branch: 123-my-branch
        meorg_output_name: True
  - repo:
      git:
        branch: 456-my-branch

name ¶

Default: base name of branch_path if an SVN repository is given; the branch name if a git repository is given; the folder name if a local path is given, optional key. An alias name used internally by benchcab for the branch. The name key also specifies the directory name of the source code when retrieving from SVN, GitHub or local.

realisations:
  - repo:
      git:
        branch: my_branch
    name: my_feature # (1)

Checkout the branch in the directory src/my_feature

build_script ¶

Default: unset, optional key. The path to a custom shell script to build the code in that branch, relative to the repository root directory. Note: any lines in the provided shell script that call the environment modules API will be ignored. To specify modules to use for the build script, please specify them using the modules key.

realisations:
  - repo:
      git:
        branch: my_branch
    build_script: offline/build.sh

install_dir ¶

Default: unset, optional key. The path to the directory containing the installed executables relative to the project root directory of the CABLE repository. If specified, benchcab will look for executables in this directory when building up the run directories.

realisations:
  - repo:
      git:
        branch: my_branch
    install_dir: path/to/bin/directory

patch ¶

Default: unset, optional key. Branch-specific namelist settings for cable.nml. Settings specified in patch get "patched" to the base namelist settings used for both branches. Any namelist settings specified here will overwrite settings defined in the default namelist file and in the science configurations. This means these settings will be set as stipulated in the patch for this branch for all science configurations run by benchcab.

The patch key must be a dictionary-like data structure that is compliant with the f90nml python package.

To specify a boolean namelist option, one needs to use the Python booleans, True and False, not the Fortran booleans

realisations:
  - repo:
      git:
        branch: my_branch
    patch:  # (1)
      cable:
        cable_user:
          FWSOIL_SWITCH: "Lai and Ktaul 2000"
          litter: True # (2)

Sets FWSOIL_SWITCH to "Lai and Ktaul 2000" for all science configurations for this branch
The Python boolean will be translated to a Fortran boolean when writing to the namelist file

patch_remove ¶

Default: unset, _optional key. Specifies branch-specific namelist settings to be removed from the cable.nml namelist settings. When the patch_remove key is specified, the specified namelists are removed from all namelist files for this branch for all science configurations run by benchcab. When specifying a namelist parameter in patch_remove, the value of the namelist parameter is ignored.

The patch_remove key must be a dictionary-like data structure that is compliant with the f90nml python package.

realisations:
  - repo:
      git:
        branch: my_branch
    patch_remove:
      cable:
        soilparmnew: nil # (1)

The value is ignored and does not have to be a possible value for the namelist option.

science_configurations¶

Default: unset, optional key. User defined science configurations. Science configurations that are specified here will replace the default science configurations. In the output filenames, each configuration is identified with S<N> where N is an integer starting from 0 for the first listed configuration and increasing by 1 for each subsequent configuration.

To specify a boolean namelist option, one needs to use the Python booleans, True and False, not the Fortran booleans

science_configurations: [
  { # S0 configuration
    cable: {
      cable_user: {
        GS_SWITCH: "medlyn",
        FWSOIL_SWITCH: "Haverd2013",
        litter: True # (1)
      }
    }
  },
  { # S1 configuration
    cable: {
      cable_user: {
        GS_SWITCH: "leuning",
        FWSOIL_SWITCH: "Haverd2013"
      }
    }
  }
]

The Python boolean will be translated to a Fortran boolean when writing the namelist file.

codecov¶

Default: False, _optional key. Specifies whether to build benchcab with code-coverage flags, which can then be used in post-run analysis (benchcab gen_codecov).

codecov:
  true

meorg_bin¶

Default: False, _optional key. Specifies the absolute system path to the ME.org client executable. In the absence of this key it will be inferred from the same directory as benchcab should meorg_output_name be set in realisations above.

meorg_bin: /path/to/meorg