WfExS-backend config

Contents

WfExS-backend config#

Title: WfExS-backend config

Type

object

Required

No

Additional properties

[Not allowed]

Description: WfExS-backend configuration file (EOSC-Life Demonstrator 7 JSON Schemas)

Property

Pattern

Type

Deprecated

Definition

Title/Description

- cacheDir

No

string

No

-

Caching directory

- crypt4gh

No

object

No

-

Installation Crypt4GH key setup

- tools

No

object

No

-

External tools configuration block

- workDir

No

string

No

-

Working directory

- fetchers-setup

No

object

No

-

Fetchers parameters setup

1. Property WfExS-backend config > cacheDir#

Title: Caching directory

Type

string

Required

No

Description: Directory where all the cache-able content will be hold. This directory can be removed, as its contents should be available outside. When it is not set, a temporary directory is created for the session, being destroyed when the program finishes.

Restrictions

Min length

1

2. Property WfExS-backend config > crypt4gh#

Title: Installation Crypt4GH key setup

Type

object

Required

No

Additional properties

[Not allowed]

Description: WfExS-backend needs an encryption key for several tasks, like encrypting and decrypting random keys of encrypted working directories. When this block does not exist, WfExS-backend.py creates the installation’s keys, and updates the configuration file

Property

Pattern

Type

Deprecated

Definition

Title/Description

+ key

No

string

No

-

Secret key installation file

+ passphrase

No

string

No

-

Passphrase used to work with the secret key

+ pub

No

string

No

-

Public key installation file

2.1. Property WfExS-backend config > crypt4gh > key#

Title: Secret key installation file

Type

string

Required

Yes

Description: The path to the Crypt4GH secret key file used by this installation. If the path is relative, the directory where the configuration file resides is used for the resolution

Restrictions

Min length

1

2.2. Property WfExS-backend config > crypt4gh > passphrase#

Title: Passphrase used to work with the secret key

Type

string

Required

Yes

Description: Passphrase which has to be used to work with the secret key

Restrictions

Min length

0

2.3. Property WfExS-backend config > crypt4gh > pub#

Title: Public key installation file

Type

string

Required

Yes

Description: The path to the Crypt4GH public key file used by this installation. If the path is relative, the directory where the configuration file resides is used for the resolution

Restrictions

Min length

1

3. Property WfExS-backend config > tools#

Title: External tools configuration block

Type

object

Required

No

Additional properties

[Not allowed]

Description: External tools configuration block

Property

Pattern

Type

Deprecated

Definition

Title/Description

- containerType

No

enum (of string)

No

-

Default container technology type in this installation

- engineMode

No

enum (of string)

No

-

Workflow engine invocation mode

- encrypted_fs

No

object

No

-

Working directory FUSE encrypted FS configuration block

- commands

No

array of object

No

-

Custom command paths

- gitCommand

No

string

No

-

Git client path

- javaCommand

No

string

No

-

Java path

- singularityCommand

No

string

No

-

Singularity client path

- dockerCommand

No

string

No

-

Docker client path

- podmanCommand

No

string

No

-

Podman client path

- dotCommand

No

string

No

-

‘dot’ client path

- staticBashCommand

No

string

No

-

Static bash command (used in singularity based Nextflow engine executions)

- staticPsCommand

No

string

No

-

Static ps command (used in Nextflow engine executions to assure metrics gathering works even in containers without /bin/ps command)

- nextflow

No

object

No

-

-

- cwl

No

object

No

-

-

- ^.+Command$

Yes

string

No

-

Path to a command

3.1. Property WfExS-backend config > tools > containerType#

Title: Default container technology type in this installation

Type

enum (of string)

Required

No

Default

"singularity"

Description: Type of container technology to be used by default when any workflow is launched using this installation. Supported types are:

  • Singularity (default).

  • Docker.

  • Podman

  • No containerisation technology (discouraged) Encrypted working directories are unsupported when Docker or Podman are used due technological limitations

Must be one of:

  • “singularity”

  • “docker”

  • “podman”

  • “none”

3.2. Property WfExS-backend config > tools > engineMode#

Title: Workflow engine invocation mode

Type

enum (of string)

Required

No

Default

"local"

Description: Most of workflow engines are usually available both as installable executables and as containers, but when they are used inside a container usually do not support running containerised jobs, unless a very careful setup is done. Currently, WfExS-backend only partially supports Nextflow in its docker-in-docker mode.

Must be one of:

  • “local”

  • “docker”

3.3. Property WfExS-backend config > tools > encrypted_fs#

Title: Working directory FUSE encrypted FS configuration block

Type

object

Required

No

Additional properties

[Not allowed]

Property

Pattern

Type

Deprecated

Definition

Title/Description

- type

No

enum (of string)

No

-

Type of encrypted FS

- command

No

string

No

-

Path to encryption mount program

- fusermount_command

No

string

No

-

Path to unmount command

- idle

No

integer

No

-

Idle minutes before autoumount

3.3.1. Property WfExS-backend config > tools > encrypted_fs > type#

Title: Type of encrypted FS

Type

enum (of string)

Required

No

Default

"gocryptfs"

Description: When an encrypted working directory is needed, the type of encrypted FS to be used is set up through this key. Currently, both encfs (default) and gocryptfs (recommended) are supported

Must be one of:

  • “encfs”

  • “gocryptfs”

3.3.2. Property WfExS-backend config > tools > encrypted_fs > command#

Title: Path to encryption mount program

Type

string

Required

No

Description: When this key is set, it overrides the default command to be used when an encrypted directory has to be created or mounted

3.3.3. Property WfExS-backend config > tools > encrypted_fs > fusermount_command#

Title: Path to unmount command

Type

string

Required

No

Default

"fusermount"

Description: Path to unmounting command to be used, being ‘fusermount’ by default

3.3.4. Property WfExS-backend config > tools > encrypted_fs > idle#

Title: Idle minutes before autoumount

Type

integer

Required

No

Default

5

Description: Number of minutes before an idle, mounted encrypted directory will automatically unmount

3.4. Property WfExS-backend config > tools > commands#

Title: Custom command paths

Type

array of object

Required

No

Description: Different modules depend on locatable programs, like git, docker, java or similar. These declarations are used to provide WfExS either custom paths or custom names. These declarations take precedence over gitCommand, javaCommand and similar ones

Array restrictions

Min items

1

Max items

N/A

Items unicity

True

Additional items

False

Tuple validation

See below

Each item of this array must be

Description

commands items

-

3.4.1. WfExS-backend config > tools > commands > commands items#

Type

object

Required

No

Additional properties

[Any type: allowed]

Property

Pattern

Type

Deprecated

Definition

Title/Description

+ key

No

string

No

-

The command key

+ path

No

string

No

-

The command path
3.4.1.1. Property WfExS-backend config > tools > commands > commands items > key#

Title: The command key

Type

string

Required

Yes

Description: Internal symbolic name used to represent this command

3.4.1.2. Property WfExS-backend config > tools > commands > commands items > path#

Title: The command path

Type

string

Required

Yes

Description: Either the absolute path or PATH findable program name

Restrictions

Min length

1

3.5. Property WfExS-backend config > tools > gitCommand#

Title: Git client path

Type

string

Required

No

Default

"git"

Description: Git is used to materialize workflows being hosted at git repositories, like GitHub. This key sets up custom paths to git command

3.6. Property WfExS-backend config > tools > javaCommand#

Title: Java path

Type

string

Required

No

Default

"java"

Description: Java is needed to run Nextflow and future workflow engines. This key sets up custom paths to java installations

3.7. Property WfExS-backend config > tools > singularityCommand#

Title: Singularity client path

Type

string

Required

No

Default

"singularity"

Description: Singularity is used when containerType is ‘singularity’. This key sets up custom paths to singularity command

3.8. Property WfExS-backend config > tools > dockerCommand#

Title: Docker client path

Type

string

Required

No

Default

"docker"

Description: Docker is used when containerType is ‘docker’. This key sets up custom paths to docker command

3.9. Property WfExS-backend config > tools > podmanCommand#

Title: Podman client path

Type

string

Required

No

Default

"podman"

Description: Podman is used when containerType is ‘podman’. This key sets up custom paths to podman command

3.10. Property WfExS-backend config > tools > dotCommand#

Title: ‘dot’ client path

Type

string

Required

No

Default

"dot"

Description: ‘dot’ is used when the workflow engine generated diagram is in DOT format, to translate it to either PNG or SVG when the diagram is going to be embedded in an RO-Crate

3.11. Property WfExS-backend config > tools > staticBashCommand#

Title: Static bash command (used in singularity based Nextflow engine executions)

Type

string

Required

No

Default

"bash.static"

Description: There is a bug in some bash versions which make them unsuitable to run the trace machinery from Nextflow, as the trace machinery enters in a live lock. As the images containing these faulty bash versions cannot be changed, a ‘monkey patch’ solution where an external, static bash version is injected on workflow execution is used. The injected static bash is found through this key, which is searched on PATH variable when it is not a full path.

3.12. Property WfExS-backend config > tools > staticPsCommand#

Title: Static ps command (used in Nextflow engine executions to assure metrics gathering works even in containers without /bin/ps command)

Type

string

Required

No

Default

"ps.static"

Description: Nextflow uses a bash wrapper to gather process execution statistics. One of the binaries used by the wrapper is ps. Some containers (for instance, r-base:latest) do not contain such binary. As the images which do not contain ps cannot be changed, a ‘monkey patch’ solution where an external, static ps version is injected on workflow execution is used. The injected static ps is found through this key, which is searched on PATH variable when it is not a full path.

3.13. Property WfExS-backend config > tools > nextflow#

Type

object

Required

No

Additional properties

[Any type: allowed]

Property

Pattern

Type

Deprecated

Definition

Title/Description

- dockerImage

No

string

No

-

Image for docker-in-docker mode

- version

No

string

No

-

Nextflow’s version

- maxRetries

No

integer

No

-

Retries in docker mode

- maxProcesses

No

Combination

No

-

(deprecated) Max number of CPUs

- maxCPUs

No

Combination

No

-

Max number of CPUs

- maxTaskDuration

No

Combination

No

-

Max task duration

3.13.1. Property WfExS-backend config > tools > nextflow > dockerImage#

Title: Image for docker-in-docker mode

Type

string

Required

No

Default

"nextflow/nextflow"

Description: (unfinished) When engineMode is docker, the name of the image to be fetched and used. The used tag will depend on the workflow’s metadata, being by default the version

3.13.2. Property WfExS-backend config > tools > nextflow > version#

Title: Nextflow’s version

Type

string

Required

No

Default

"19.04.1"

Description: Version of Nextflow engine to be used when workflow’s metadata does not provide hints about minimal version needed.

3.13.3. Property WfExS-backend config > tools > nextflow > maxRetries#

Title: Retries in docker mode

Type

integer

Required

No

Default

5

Description: Retries when engineMode is docker. Retries system was introduced when using docker-in-docker pattern because an insidious bug happens sometimes. See https://forums.docker.com/t/any-known-problems-with-symlinks-on-bind-mounts/32138

Restrictions

Minimum

≄ 0

3.13.4. Property WfExS-backend config > tools > nextflow > maxProcesses#

Title: (deprecated) Max number of CPUs

Type

combining

Required

No

Additional properties

[Any type: allowed]

Description: Number of CPUs to be used by Nextflow. When this key has an explicit value of null, it depends on Nextflow criteria, which tries creating as many processes as available CPUs, spawning jobs in parallel. Not declaring it, or declaring and explicit value, imposes a limitation in the number of concurrent processes

One of(Option)

item 0

item 1
3.13.4.1. Property WfExS-backend config > tools > nextflow > maxProcesses > oneOf > item 0#

Type

integer

Required

No

Default

4

Restrictions

Minimum

≄ 1
3.13.4.2. Property WfExS-backend config > tools > nextflow > maxProcesses > oneOf > item 1#

Type

null

Required

No

3.13.5. Property WfExS-backend config > tools > nextflow > maxCPUs#

Title: Max number of CPUs

Type

combining

Required

No

Additional properties

[Any type: allowed]

Description: Number of CPUs to be used by Nextflow. When this key has an explicit value of null, it depends on Nextflow criteria, which tries creating as many processes as available CPUs, spawning jobs in parallel. Not declaring it, or declaring and explicit value, imposes a limitation in the number of concurrent processes

One of(Option)

item 0

item 1
3.13.5.1. Property WfExS-backend config > tools > nextflow > maxCPUs > oneOf > item 0#

Type

integer

Required

No

Default

4

Restrictions

Minimum

≄ 1
3.13.5.2. Property WfExS-backend config > tools > nextflow > maxCPUs > oneOf > item 1#

Type

null

Required

No

3.13.6. Property WfExS-backend config > tools > nextflow > maxTaskDuration#

Title: Max task duration

Type

combining

Required

No

Additional properties

[Any type: allowed]

Description: With this parameter it is possible to override the max task time declared for processes. This is needed for workflows which are not so customizable

One of(Option)

item 0

item 1
3.13.6.1. Property WfExS-backend config > tools > nextflow > maxTaskDuration > oneOf > item 0#

Type

string

Required

No

Restrictions

Must match regular expression

^[0-9]+.*$ Test
3.13.6.2. Property WfExS-backend config > tools > nextflow > maxTaskDuration > oneOf > item 1#

Type

null

Required

No

3.14. Property WfExS-backend config > tools > cwl#

Type

object

Required

No

Additional properties

[Any type: allowed]

Property

Pattern

Type

Deprecated

Definition

Title/Description

- version

No

string

No

-

cwltool’s version

3.14.1. Property WfExS-backend config > tools > cwl > version#

Title: cwltool’s version

Type

string

Required

No

Default

"3.1.20210628163208"

Description: Version of cwltool engine to be used. WfExS is not currently guessing the minimal needed version, so it is either the value set up in this key or the default one

Restrictions

Min length

1

3.15. Pattern Property WfExS-backend config > tools > Path to a command#

All properties whose name matches the regular expression ^.+Command$ (Test) must respect the following conditions

Title: Path to a command

Type

string

Required

No

Description: This is a wildcard declaration to allow setting up custom paths to commands needed by several parts of the code

Restrictions

Min length

1

4. Property WfExS-backend config > workDir#

Title: Working directory

Type

string

Required

No

Description: Directory where all the working directories are going to be created. When it is not set, a temporary directory is created, which will be removed when the program finishes (which avoids inspecting the working directory after the program has finished).

Restrictions

Min length

1

5. Property WfExS-backend config > fetchers-setup#

Title: Fetchers parameters setup

Type

object

Required

No

Additional properties

[Any type: allowed]

Description: Some fetchers could need customizations at the configuration level, like limiting throughput or setting up some proxy

Property

Pattern

Type

Deprecated

Definition

Title/Description

- ^[a-z][a-z0-9+.-]*$

Yes

object

No

-

Scheme fetcher setup

5.1. Pattern Property WfExS-backend config > fetchers-setup > Scheme fetcher setup#

All properties whose name matches the regular expression ^[a-z][a-z0-9+.-]*$ (Test) must respect the following conditions

Title: Scheme fetcher setup

Type

object

Required

No

Additional properties

[Any type: allowed]

Description: Some scheme fetchers could need customizations which depend on local WfExS installation environment. This is the place where to pass that

Generated using json-schema-for-humans on 2024-12-03 at 23:43:14 +0000

Contents