1.8 Dynamic naming

Learning objectives

Understand the purpose of dynamic output naming
Apply input variables to dynamically generate output file names
Distinguish between single and double quotes when working with variables in Groovy syntax

When we run bioinformatics pipelines, it is common to process and identify results by the sample name or by other metadata.

Currently, the outputs of the SAYHELLO and CONVERTTOUPPER processes are being saved as output.txt and upper.txt, respectively. In some situations this would be fine. However, it would help to identify the outputs by naming these dynamically.

Let's get tricky and name your output files dynamically.

1.8.1 Expressing process outputs dynamically

When an output file name needs to be expressed dynamically, it is possible to define it using a dynamic string that references values defined in the input declaration block or in the script global context.

For example, the SAYHELLO input value greeting can be used to help name the output file.

process SAYHELLO {
    publishDir 'results'

    input:
    val greeting

    output:
    path "${greeting}.txt"

    script:
    """
    echo '$greeting' > ${greeting}.txt
    """
}

Curly brackets {} have been used to wrap greeting in the output and script block so it is interpreted as a variable as a part of a file name.

There is an important difference between single-quoted (') and double-quoted (")strings. Double-quoted strings support variable interpolations while single-quoted strings do not.

Exercise

Update the SAYHELLO and CONVERTTOUPPER process to use dynamic output names.

Warning

It's difficult to name a file with a space. Use a simple greeting without spaces, e.g., "Hello", when testing your pipeline.

Solution

hello-world.nf

// Use echo to print 'Hello World!' and redirect to output.txt
process SAYHELLO {
    publishDir 'results'

    input:
    val greeting

    output:
    path "${greeting}.txt"

    script:
    """
    echo '$greeting' > ${greeting}.txt
    """
}

// Use tr to convert lowercase letters to upper case letters and save as upper.txt
process CONVERTTOUPPER {
    publishDir 'results'

    input:
        path input_file

    output:
        path "upper_${input_file}"

    script:
    """
    cat $input_file | tr '[a-z]' '[A-Z]' > upper_${input_file}
    """
}

workflow {

    // Set default greeting
    params.greeting = "Hello"

    // Create a channel for inputs
    greeting_ch = Channel.of(params.greeting)

    // Emit a greeting
    SAYHELLO(greeting_ch)

    // Convert the greeting to uppercase
    CONVERTTOUPPER(SAYHELLO.out)

}

Let's execute your pipeline and view the changes to see if your outputs have been named dynamically.

nextflow run hello-world.nf --greeting 'Hello'

While the output will look the same:

N E X T F L O W  ~  version 24.10.2
Launching `hello-world.nf` [mighty_murdock] DSL2 - revision: 80e92a677c
executor >  local (2)
[ef/b99a2f] SAYHELLO (1)       [100%] 1 of 1 ✔
[cd/c8cf1b] CONVERTTOUPPER (1) [100%] 1 of 1 ✔

You should now see some new files in your results folder:

Hello.txt
upper_Hello.txt

Summary

In this lesson, we made our pipeline outputs more informative by:

Using dynamic names to name outputs such as Hello.txt
Updated both the SAYHELLO and CONVERTTOUPPER processes to use interpolated file names
Applied double-quoted strings and curly braces {} for variable expansion in both script and output blocks
Noted the limitation of using file names with spaces, reinforcing the importance of simple input values for filenames