CLI
WfExS (workflow execution service) backend 0.99.0
usage: WfExS-backend.py [-h] [--log-file LOGFILENAME] [-q] [-v] [-d]
[-L LOCALCONFIGFILENAME] [--cache-dir CACHEDIR] [-V]
[--full-help]
{init,cache,staged-workdir,export,list-fetchers,list-exporters,list-licences,config-validate,stage,re-stage,mount-workdir,export-stage,offline-execute,execute,export-results,export-crate}
...
Named Arguments
- --log-file
Store messages in a file instead of using standard error and standard output
- -q, --quiet
Only show engine warnings and errors
- -v, --verbose
Show verbose (informational) messages
- -d, --debug
Show debug messages (use with care, as it can disclose passphrases and passwords)
- -L, --local-config
Local installation configuration file (can also be set up through WFEXS_CONFIG_FILE environment variable)
Default: “/home/docs/checkouts/readthedocs.org/user_builds/wfexs-backend/checkouts/latest/docs/source/wfexs_config.yml”
- --cache-dir
Caching directory
- -V, --version
show program’s version number and exit
- --full-help
It returns full help
Default: False
commands
Command to run. It must be one of these
- command
Possible choices: init, cache, staged-workdir, export, list-fetchers, list-exporters, list-licences, config-validate, stage, re-stage, mount-workdir, export-stage, offline-execute, execute, export-results, export-crate
Sub-commands
init
Init local setup
WfExS-backend.py init [-h]
cache
Cache handling subcommands
WfExS-backend.py cache [-h] [-r] [--cascade] [-g]
{ls,status,inject,fetch,rm,validate}
{input,ro-crate,ga4gh-trs,workflow}
[cache_command_args ...]
Positional Arguments
- cache_command
Possible choices: ls, status, inject, fetch, rm, validate
Cache command to perform
ls List the cache entries status Show the cache entries metadata inject Inject a new entry in the cache fetch Fetch a new cache entry, giving as input both the URI and optionally both a security context file and a security context name rm Remove an entry from the cache validate Validate the consistency of the cache
- cache_type
Possible choices: input, ro-crate, ga4gh-trs, workflow
Cache type to perform the cache command
input Cached or injected inputs ro-crate Cached RO-Crates (usually from WorkflowHub) ga4gh-trs Cached files from tools described at GA4GH TRS repositories workflow Cached workflows, which come from a git repository
- cache_command_args
Optional cache element names
Named Arguments
- -r
Try doing the operation recursively (i.e. both metadata and data)
Default: False
- --cascade
Try doing the operation in cascade (including the URIs which resolve to other URIs)
Default: False
- -g, --glob
Given cache element names are globs
Default: False
staged-workdir
Staged working directories handling subcommands
WfExS-backend.py staged-workdir [-h] [--orcid ORCIDS]
[--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
[--inputs] [--outputs] [--workflow]
[--containers] [--prospective] [--full]
[--licence LICENCES] [--crate-pid CRATE_PID]
[-g]
{offline-exec,ls,mount,rm,shell,status,create-staged-crate,create-prov-crate}
[staged_workdir_command_args ...]
Positional Arguments
- staged_workdir_command
Possible choices: offline-exec, ls, mount, rm, shell, status, create-staged-crate, create-prov-crate
Staged working directory command to perform
offline-exec Offline execute the staged instances which match the input pattern ls List the staged instances
It shows the instance id, nickname, encryption and whether they are damaged
mount Mount the staged instances which match the input pattern rm Removes the staged instances which match the input pattern shell Launches a command in the workdir
First parameter is either the staged instance id or the nickname. It launches the command specified after the id. If there is no additional parameters, it launches a shell in the mounted working directory of the instance
status Shows staged instances status create-staged-crateIt creates an RO-Crate from the prospective provenance create-prov-crateIt creates an RO-Crate from the retrospective provenance (after a workflow execution)
- staged_workdir_command_args
Optional staged working directory element names
Named Arguments
- --orcid
ORCID(s) of the person(s) staging, running or exporting the workflow scenario
Default: []
- -g, --glob
Given staged workflow names are globs
Default: False
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
ro-crate-payload
What to include in the RO-Crate
- --inputs
Should the RO-Crate contain a inputs copy (of everything)?
Default: []
- --outputs
Should the RO-Crate contain a outputs copy (of everything)?
Default: []
- --workflow
Should the RO-Crate contain a workflow copy (of everything)?
Default: []
- --containers
Should the RO-Crate contain a containers copy (of everything)?
Default: []
- --prospective
Should the RO-Crate contain a prospective copy (of everything)?
Default: []
- --full
Should the RO-Crate contain a full copy (of everything)?
Default: []
- --licence
Licence(s) to attach to the generated RO-Crate
Default: []
- --crate-pid
Permanent identifier to embed within the generated RO-Crate metadata, like a pre-generated DOI
export
Staged working directories export subcommands
WfExS-backend.py export [-h] [-Z SECURITYCONTEXTSCONFIGFILENAME]
[-E EXPORTSCONFIGFILENAME] [--orcid ORCIDS]
[--public-key-file PUBLIC_KEY_FILES]
[--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
-J WORKFLOWWORKINGDIRECTORY [--licence LICENCES]
{ls,run} [export_contents_command_args ...]
Positional Arguments
- export_contents_command
Possible choices: ls, run
Export operations from staged working directory to perform
ls List the public identifiers obtained from previous export actions run Run the different export actions, pushing the exported content and gathering the obtained permanent / public identifiers
- export_contents_command_args
Optional export names
Named Arguments
- -Z, --creds-config
Configuration file, describing security contexts, which hold credentials and similar
- -E, --exports-config
Configuration file, describing exports which can be done
- --orcid
ORCID(s) of the person(s) staging, running or exporting the workflow scenario
Default: []
- --public-key-file
This parameter switches on secure processing. Path to the public key(s) to be used to encrypt the working directory
Default: []
- -J, --staged-job-dir
Already staged job directory
- --licence
Licence(s) to attach to the exported contents
Default: []
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
list-fetchers
List the supported fetchers / schemes
WfExS-backend.py list-fetchers [-h]
list-exporters
List the supported export plugins
WfExS-backend.py list-exporters [-h]
list-licences
List the documented licences, both embedded and fetched from SPDX release 3.23
WfExS-backend.py list-licences [-h]
config-validate
Validate the configuration files to be used for staging and execution
WfExS-backend.py config-validate [-h] -W WORKFLOWCONFIGFILENAME
[-Z SECURITYCONTEXTSCONFIGFILENAME]
Named Arguments
- -W, --workflow-config
Configuration file, describing workflow and inputs
- -Z, --creds-config
Configuration file, describing security contexts, which hold credentials and similar
stage
Prepare the staging (working) directory for workflow execution, fetching dependencies and contents
WfExS-backend.py stage [-h] -W WORKFLOWCONFIGFILENAME
[-Z SECURITYCONTEXTSCONFIGFILENAME]
[-n NICKNAME_PREFIX] [--orcid ORCIDS]
[--public-key-file PUBLIC_KEY_FILES]
[--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
Named Arguments
- -W, --workflow-config
Configuration file, describing workflow and inputs
- -Z, --creds-config
Configuration file, describing security contexts, which hold credentials and similar
- -n, --nickname-prefix
Nickname prefix to be used on staged workdir creation
- --orcid
ORCID(s) of the person(s) staging, running or exporting the workflow scenario
Default: []
- --public-key-file
This parameter switches on secure processing. Path to the public key(s) to be used to encrypt the working directory
Default: []
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
re-stage
Prepare a new staging (working) directory for workflow execution, repeating the fetch of dependencies and contents
WfExS-backend.py re-stage [-h] [-Z SECURITYCONTEXTSCONFIGFILENAME]
[-n NICKNAME_PREFIX] [--orcid ORCIDS]
[--public-key-file PUBLIC_KEY_FILES]
[--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
-J WORKFLOWWORKINGDIRECTORY
Named Arguments
- -Z, --creds-config
Configuration file, describing security contexts, which hold credentials and similar
- -n, --nickname-prefix
Nickname prefix to be used on staged workdir creation
- --orcid
ORCID(s) of the person(s) staging, running or exporting the workflow scenario
Default: []
- --public-key-file
This parameter switches on secure processing. Path to the public key(s) to be used to encrypt the working directory
Default: []
- -J, --staged-job-dir
Already staged job directory
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
mount-workdir
Mount the encrypted staging directory on secure staging scenarios
WfExS-backend.py mount-workdir [-h] [--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
-J WORKFLOWWORKINGDIRECTORY
Named Arguments
- -J, --staged-job-dir
Already staged job directory
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
export-stage
Export the staging directory as an RO-Crate
WfExS-backend.py export-stage [-h] [--orcid ORCIDS]
[--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
-J WORKFLOWWORKINGDIRECTORY [--inputs]
[--outputs] [--workflow] [--containers]
[--prospective] [--full] [--licence LICENCES]
[--crate-pid CRATE_PID]
Named Arguments
- --orcid
ORCID(s) of the person(s) staging, running or exporting the workflow scenario
Default: []
- -J, --staged-job-dir
Already staged job directory
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
ro-crate-payload
What to include in the RO-Crate
- --inputs
Should the RO-Crate contain a inputs copy (of everything)?
Default: []
- --outputs
Should the RO-Crate contain a outputs copy (of everything)?
Default: []
- --workflow
Should the RO-Crate contain a workflow copy (of everything)?
Default: []
- --containers
Should the RO-Crate contain a containers copy (of everything)?
Default: []
- --prospective
Should the RO-Crate contain a prospective copy (of everything)?
Default: []
- --full
Should the RO-Crate contain a full copy (of everything)?
Default: []
- --licence
Licence(s) to attach to the generated RO-Crate
Default: []
- --crate-pid
Permanent identifier to embed within the generated RO-Crate metadata, like a pre-generated DOI
offline-execute
Execute an already prepared workflow in the staging directory
WfExS-backend.py offline-execute [-h] [--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
-J WORKFLOWWORKINGDIRECTORY
Named Arguments
- -J, --staged-job-dir
Already staged job directory
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
execute
Execute the stage + offline-execute + export steps
WfExS-backend.py execute [-h] -W WORKFLOWCONFIGFILENAME
[-Z SECURITYCONTEXTSCONFIGFILENAME]
[-E EXPORTSCONFIGFILENAME] [-n NICKNAME_PREFIX]
[--orcid ORCIDS] [--public-key-file PUBLIC_KEY_FILES]
[--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
[--inputs] [--outputs] [--workflow] [--containers]
[--prospective] [--full] [--licence LICENCES]
[--crate-pid CRATE_PID]
Named Arguments
- -W, --workflow-config
Configuration file, describing workflow and inputs
- -Z, --creds-config
Configuration file, describing security contexts, which hold credentials and similar
- -E, --exports-config
Configuration file, describing exports which can be done
- -n, --nickname-prefix
Nickname prefix to be used on staged workdir creation
- --orcid
ORCID(s) of the person(s) staging, running or exporting the workflow scenario
Default: []
- --public-key-file
This parameter switches on secure processing. Path to the public key(s) to be used to encrypt the working directory
Default: []
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
ro-crate-payload
What to include in the RO-Crate
- --inputs
Should the RO-Crate contain a inputs copy (of everything)?
Default: []
- --outputs
Should the RO-Crate contain a outputs copy (of everything)?
Default: []
- --workflow
Should the RO-Crate contain a workflow copy (of everything)?
Default: []
- --containers
Should the RO-Crate contain a containers copy (of everything)?
Default: []
- --prospective
Should the RO-Crate contain a prospective copy (of everything)?
Default: []
- --full
Should the RO-Crate contain a full copy (of everything)?
Default: []
- --licence
Licence(s) to attach to the generated RO-Crate
Default: []
- --crate-pid
Permanent identifier to embed within the generated RO-Crate metadata, like a pre-generated DOI
export-results
Export the results to a remote location, gathering their public ids
WfExS-backend.py export-results [-h] [--orcid ORCIDS]
[--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
-J WORKFLOWWORKINGDIRECTORY
[--licence LICENCES]
Named Arguments
- --orcid
ORCID(s) of the person(s) staging, running or exporting the workflow scenario
Default: []
- -J, --staged-job-dir
Already staged job directory
- --licence
Licence(s) to attach to the exported contents
Default: []
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
export-crate
Export an already executed workflow in the staging directory as an RO-Crate
WfExS-backend.py export-crate [-h] [--orcid ORCIDS]
[--private-key-file PRIVATE_KEY_FILE]
[--private-key-passphrase-envvar PRIVATE_KEY_PASSPHRASE_ENVVAR]
-J WORKFLOWWORKINGDIRECTORY [--inputs]
[--outputs] [--workflow] [--containers]
[--prospective] [--full] [--licence LICENCES]
[--crate-pid CRATE_PID]
Named Arguments
- --orcid
ORCID(s) of the person(s) staging, running or exporting the workflow scenario
Default: []
- -J, --staged-job-dir
Already staged job directory
secure workdir arguments
Private key and passphrase to access secured working directories
- --private-key-file
This parameter passes the name of the file containing the private key needed to unlock an encrypted working directory.
- --private-key-passphrase-envvar
This parameter passes the name of the environment variable containing the passphrase needed to decrypt the private key needed to unlock an encrypted working directory.
Default: “”
ro-crate-payload
What to include in the RO-Crate
- --inputs
Should the RO-Crate contain a inputs copy (of everything)?
Default: []
- --outputs
Should the RO-Crate contain a outputs copy (of everything)?
Default: []
- --workflow
Should the RO-Crate contain a workflow copy (of everything)?
Default: []
- --containers
Should the RO-Crate contain a containers copy (of everything)?
Default: []
- --prospective
Should the RO-Crate contain a prospective copy (of everything)?
Default: []
- --full
Should the RO-Crate contain a full copy (of everything)?
Default: []
- --licence
Licence(s) to attach to the generated RO-Crate
Default: []
- --crate-pid
Permanent identifier to embed within the generated RO-Crate metadata, like a pre-generated DOI