Welcome!
We developed a Nextflow DSL2 workflow template and this user guide to aid beginners in developing their own Nextflow workflows. Here, we guide you through how to use our workflow template to develop your own Nextflow pipelines.
Nextflow is open source and scalable workflow management software for bioinformatics. It enables the development and running of integrated, reproducible workflows consisting of multiple processes, various environment management systems, scripting languages, and software packages. While Nextflow is designed to have a minimal learning curve as it doesn’t require end users to learn new programming languages, its extensive capabilities, use of Groovy syntax, and comprehensive documentation can be overwhelming for users who aren’t well versed in programming and software development.
This user guide is under active development and will be updated progressively.
How to use this guide
This guide explains all aspects of using the workflow template, including:
1. How to use the template on GitHub
Start here for directions on how to set yourself up to use this template. This section provides step-by-step instructions to help you set up your local development environment and use the template files.
2. The template structure and components
Use the template components sections to familiarise yourself with the essential elements of the template including code base structure, directory organisation, configuration files, and how to modularise your workflow.
3. Resource configuration
The resource configuration section can help you configure the resources required for running your workflow efficiently. The guide will cover topics such as specifying CPU and memory requirements, handling parallel execution, and integrating with high-performance computing (HPC) environments.
4. Guides for customising the template
In the guides you will find various examples of scenarios you may come across when writing Nextflow code and developing your workflows. If you would like to contribute to this section, please submit a pull request.
5. A hands-on tutorial
Get a feel for the template and how to use it with the hands-on tutorial. It walks you through a practical example that demonstrates the application of the workflow template.
6. Documentation and training resources
The details contained within this user guide are limited, as such we strongly recommend you explore additional documentation and training resources. These resources serve as references for further exploration of Nextflow concepts, advanced topics, and community support.
If you are new to this template and/or Nextflow, we recommend you do the following before applying the template to your own workflow:
- Familiarise yourself with Nextflow
- Clone the template repository
- Explore the template structure as you run through the hands-on exercise
- Review the workflow structure
- Customise the resource configuration for your own needs
Who is the DSL2 template for?
The DSL2 workflow template is suitable for:
- Nextflow newcomers looking for a low barrier to entry, structured starting point, and guidance.
- Custom workflow developers looking for a simple scaffold which can be extended and modified as needed.
- Collaborative teams looking for a standard and consistent workflow code base structure.
- Scalable workflow developers looking for a scalable and reproducible solution for their data analysis and processing needs.
It is not suitable for:
- Those wishing to create and contribute to public nf-core workflows, as it is not nf-core compatible.
- Those creating simple or single-task workflows with only a few tasks and minimal complexity.
- Those needing to rapidly prototype a workflow or perform exploratory analysis.
- Those with no previous command-line and bash experience.
Acknowledgements
The Nextflow DSL2 template and accompanying materials were developed by the Sydney Informatics Hub, University of Sydney in partnership with the Australian BioCommons - Bring Your Own Data Platforms project (Australian Research Data Commons and NCRIS via Bioplatforms Australia).