Job submission with qsub

[fbartusch]

Hello,

A user on our HPC system wanted to run metaGEM. Jobs are submitted with qsub to our PBS/Torque scheduling. As the pipeline only supports SLURM, I created a fork and implemented submission to our cluster. I commented out the sbatch commands and replaced them with suitable qsub commands, e.g.
https://github.com/fbartusch/metaGEM/blob/master/workflow/metaGEM.sh#L316

I had to add a line to the cluster_config.json for setting the queue on our cluster:
https://github.com/fbartusch/metaGEM/blob/master/config/cluster_config.json#L10

As it's a quick fix for our cluster it's not suitable for a pull request. But you could implement support for qsub by taking the few changes in the fork and some more if clauses in metaGEM.sh.

Also, I ran into the issue that I need a certain snakemake version (5.10). You have a nice script that sets up all the environments for the pipeline. Can you also install the correct snakemake version in that script and document the version needed in the README? I think that would really help new users.

[franciscozorrilla]

Hey @fbartusch , thank you for the helpful suggestions.

Indeed there have been some users asking for PBS/Torque support, but since I have not had access to any such clusters I have not been able to test and troubleshoot myself. I will point future PBS/Torque users towards your fork as a reference point.

Regarding Snakemake version, could you elaborate on the method used to install metaGEM dependencies that resulted in the incorrect version being installed?

Btw, if you mamba/conda install metagem as outlined in the main README you should get the correct Snakemake version installed, as you can see from the bioconda recipe.

mamba create -n metagem -c bioconda metagem

Similarly, the env_setup.sh script uses this recipe file which should also set up the correct Snakemake version.

Please let me know if you followed any of the above installation methods and still had problems with the Snakemake version.

I have also updated the wiki documentation manual setup page, specifically adding an allowable snakemake version range, as well as the config README (ed30e9c). I will also add a badge to the main README specifying the allowable snakemake range.

[fbartusch]

Hi @franciscozorrilla,
thanks for the quick reply. I followed the snakemake documentation to install snakemake:

# This installs snakemake 7.32.3 for me
mamba create -c conda-forge -c bioconda -n snakemake snakemake

Indeed, when installing metaGEM with mamba like in the README it installs snakemake 5.31.0. As I wanted to change the code I thought I just clone the repo and use the env_setup.sh script for setting up the environment. I saw this in a tutorial and tried that out as I was totally new to metaGEM. Thanks for clarifying that.