previous episode

Introduction to NCI Gadi

next episode

Software

Overview

Teaching: 5 min
Exercises: 0 min
Questions

  • Global versus local software

  • Make your own local software compatible with module commands

Objectives

  • How to make your locally installed software compatible with module commands

Global versus local software

Globally installed software (available to all users) is intalled by NCI in /apps. The module commands you are familiar with on Artemis can be used on Gadi, eg

module avail
module list
module load <software/version>
module unload <software/version>
module purge


NCI will only globally install software if it is requested by numerous projects - the requirements are quite strict. Users are encouraged to install their own software locally. NCI can provide assistance if you run into difficulties - submit a help request by emailing help@nci.org.au. For installations that may run for longer than 30 minutes, submit the installation commands as a job on copyq, as commands executing on the login node are terminated after 30 minutes.


In the example below, we install apps into /scratch. There are pros and cons:

  • CON: /scratch has a purge policy - your apps are not permanent residents here! Take backups to RDS!
  • PRO: there is ample space (unlike /home)
  • PRO: all users in your project can use the same apps

/home is a reliable (backed up, persistent) location for your locally installed apps, however no other user in your group will be able to use them. Also, some installations consume a lot of disk and your quota in /home is only 10 GB.

/g/data is another option - accessible by all in the project, not backed up, but not purged. However, not all groups have /g/data access as the use of this filesystem incurs a fee.

Carefully consider the pros and cons of where you install your apps, and if you choose to use /scratch, ensure you backup your apps directory to RDS and actively monitor for any emails you may receive about impending purges of that shared filesystem.


Using module commands with locally installed software

Locally installed software can also be loaded with the module load command, by following these setup steps:

  • Install the software into /scratch/<project>/apps
    • Or your peferred alternate location, but be sure your /dir/apps direcotry is consistent and updated in th steps below)
    • Rename the installation directory to <software>/<version>, eg for software ‘samtools’ version 1.11, the install directory should be /scratch/<project>/apps/samtools/1.11
  • Create a file called .base in directory /scratch/<project>/apps/Modules/modulefiles/<software>.
    • If the software you are installing is installed on Artemis, you can copy the Artemis .base file - these are stored in /usr/local/Modules/modulefiles/<software>.
    • If not, you can create a .base file following one of the examples below (replace <project> with your NCI project code).
    • The first example is for software that requires no specific dependencies or environment variables
    • The second has one dependency and a handful of required environment variables
                    
    #%Module1.0#####################################################################
    ##
    ## $name modulefile
    ##

    set ver [lrange [split [ module-info name ] / ] 1 1 ]
    set name [lrange [split [ module-info name ] / ] 0 0 ]
    set loading [module-info mode load]
    set desc [join [read [ open "/scratch/<project>/apps/Modules/modulefiles/$name/.desc" ] ] ]

    proc ModulesHelp { } {
    puts stderr "\tThis module sets the envinronment for $name v$ver"
    }

    module-whatis   "$desc (v$ver)"

    prepend-path PATH /scratch/<project>/apps/$name/$ver/bin


    #%Module1.0#####################################################################
    ##
    ## $name modulefile
    ##

    set ver [lrange [split [ module-info name ] / ] 1 1 ]
    set name [lrange [split [ module-info name ] / ] 0 0 ]
    set loading [module-info mode load]
    set desc [join [read [ open "/scratch/<project>/apps/Modules/modulefiles/$name/.desc" ] ] ]

    proc ModulesHelp { } {
    puts stderr "\tThis module sets the envinronment for $name v$ver"
    }

    module-whatis   "$desc (v$ver)"

    if { $loading && ![ is-loaded bcftools/1.3.1 ] } {
        module load bcftools/1.3.1
    }

    prepend-path --delim " " CFLAGS -I /scratch/<project>/apps/$name/$ver/include
    prepend-path --delim " " CPPFLAGS -I /scratch/<project>/apps/$name/$ver/include
    prepend-path --delim " " LDFLAGS -L /scratch/<project>/apps/$name/$ver/lib
    prepend-path LD_LIBRARY_PATH /scratch/<project>/apps/$name/$ver/lib
    prepend-path PKG_CONFIG_PATH /scratch/<project>/apps/$name/$ver/lib/pkgconfig
    prepend-path PATH /scratch/<project>/apps/$name/$ver/bin
    prepend-path MANPATH /scratch/<project>/apps/$name/$ver/share/man
    prepend-path BAM_ROOT /scratch/<project>/apps/$name/$ver/bin
  • If you have copied the .base file from Artemis, change /usr/local to /scratch/<project>/apps with the following command: sed -i 's/\/usr\/local/\/scratch\/<project>\/apps/g' .base
  • Create (or copy from Artemis) a .desc (description) file into /scratch/<project>/apps/Modules/modulefiles/<software>. This is just a one-line description of the software.
  • Link the .base file to the software version using the following command: ln -s .base <version>
    • Ensure that <version> exactly matches that used to name the directory /scratch/<project>/apps/<software>/<version>
  • To make ‘module’ commands work for locally installed software, add the following line to your .bash_profile and .bashrc files: export MODULEPATH=${MODULEPATH}:/scratch/<project>/apps/Modules/modulefiles
  • You will need to launch a new session or run source on these files for the changes to take effect


Short break

We will now take a ten minute break. The next two sections cover optimisation and some hands-on example parallel jobs, around 45 minutes total. We encourage you to stay for one or both if you can, however if not, please feel free to work through the material at your own leisure.

:raised_hand: If you have any questions, please post these in the chat and we will discuss them during the break.

:bouquet: If you are leaving the course now, we thank you for your attendance, and encourage you to take our survey - your feedback would be highly appreciated :relaxed:


Key Points

  • Unlike our Artemis support, NCI won’t (rarely) install software globally upon request, but they will assist you to perform local installations

  • You can make your locally installed software compatible with module commands by following the steps above

previous episode

next episode