db split1 issue #89
Comments
|
Could you try to submit task |
|
I am trying to rework it for slurm like this However, it keeps saying cant open smp. Is this an option? |
|
If you are using |
|
Hmm, I am confused about that. This is what happened when I tried the qsub command: The command was: |
|
I believe I got this to work with: However, I am unsure if there are further steps since I cant seem to find the output files. |
|
Hi, see here and ParallelTask to change the submit command template. |
Describe the bug
Hi I am trying to run NextPolish on Hifi pacbio sequences. I keep running into an unspecified error with db split.
My workflow is:
ls /scratch/kcl58759/Eco_pacbio_kendall/pb_css_474/cromwell-executions/pb_ccs/c7a3dc30-7f94-40de-ac16-2445f965bfad/call-export_fasta/execution/m64060_210804_174320.hifi_reads.fasta.gz > lgs.fofn
Make config file:
#creat config file run.cfg
job_type = slurm
job_prefix = nextPolish
task = best
rewrite = yes
rerun = 3
parallel_jobs = 6
multithread_jobs = 5
genome = /scratch/kcl58759/Eco_pacbio_kendall/474.Primary.Hifi.asm/474.Primary.HiFi.asm.p_ctg.fa
genome_size = auto
workdir = ./01_rundir
polish_options = -p {multithread_jobs}
[lgs_option]
lgs_fofn = ./lgs.fofn
lgs_options = -min_read_len 1k -max_depth 100
lgs_minimap2_options = -x map-ont
then I submit with
#!/bin/bash
#SBATCH --job-name=NextPolish
#SBATCH --partition=batch
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=6
#SBATCH --mem=90gb
#SBATCH --time=99:00:00
#SBATCH --output=nextpolish.out
#SBATCH --error=nextpolish.err
#SBATCH --mail-user=kcl58759@uga.edu
#SBATCH --mail-type=END,FAIL
nextPolish run.cfg
Error message
[60417 INFO] 2022-03-03 13:02:46 NextPolish start...
[60417 INFO] 2022-03-03 13:02:46 version:v1.4.0 logfile:pid60417.log.info
[60417 WARNING] 2022-03-03 13:02:46 Re-write workdir
[60417 WARNING] 2022-03-03 13:03:05 Delete task: 1 due to missing sgs_fofn.
[60417 WARNING] 2022-03-03 13:03:05 Delete task: 1 due to missing sgs_fofn.
[60417 WARNING] 2022-03-03 13:03:05 Delete task: 2 due to missing sgs_fofn.
[60417 WARNING] 2022-03-03 13:03:05 Delete task: 2 due to missing sgs_fofn.
[60417 WARNING] 2022-03-03 13:03:05 Delete task: 6 due to missing hifi_fofn.
[60417 WARNING] 2022-03-03 13:03:05 Delete task: 6 due to missing hifi_fofn.
[60417 INFO] 2022-03-03 13:03:05 scheduled tasks:
[5, 5]
[60417 INFO] 2022-03-03 13:03:05 options:
[60417 INFO] 2022-03-03 13:03:05
rerun: 3
rewrite: 1
kill: None
cleantmp: 0
task: [5, 5]
use_drmaa: 0
submit: None
job_type: sge
sgs_unpaired: 0
sgs_rm_nread: 1
parallel_jobs: 6
align_threads: 5
check_alive: None
job_id_regex: None
sgs_max_depth: 100
lgs_max_depth: 100
lgs_read_type: clr
multithread_jobs: 5
lgs_max_read_len: 0
hifi_max_depth: 100
polish_options: -p 5
lgs_min_read_len: 1k
hifi_max_read_len: 0
genome_size: 36224976
hifi_block_size: 500M
hifi_min_read_len: 1k
job_prefix: nextPolish
sgs_block_size: 500000000
lgs_block_size: 500000000
sgs_use_duplicate_reads: 0
sgs_align_options: bwa mem
hifi_minimap2_options: -x map-pb
lgs_minimap2_options: -x map-pb -t 5
lgs_fofn: /scratch/kcl58759/Eco_pacbio_kendall/./lgs.fofn
workdir: /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir
snp_phase: /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/%02d.snp_phase
snp_valid: /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/%02d.snp_valid
lgs_polish: /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/%02d.lgs_polish
kmer_count: /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/%02d.kmer_count
hifi_polish: /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/%02d.hifi_polish
score_chain: /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/%02d.score_chain
genome: /scratch/kcl58759/Eco_pacbio_kendall/474.Primary.Hifi.asm/474.Primary.HiFi.asm.p_ctg.fa
[60417 INFO] 2022-03-03 13:03:05 step 0 and task 5 start:
[60417 INFO] 2022-03-03 13:03:19 Total jobs: 2
[60417 CRITICAL] 2022-03-03 13:03:19 Command 'qsub -pe smp 5 -l vf=2.5G -o /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/00.lgs_polish/01.db_split.sh.work/db_split1/nextPolish.sh.o -e /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/00.lgs_polish/01.db_split.sh.work/db_split1/nextPolish.sh.e /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/00.lgs_polish/01.db_split.sh.work/db_split1/nextPolish.sh' returned non-zero exit status 1, error info: .
Traceback (most recent call last):
File "/apps/eb/NextPolish/1.4.0-GCCcore-8.3.0-Python-3.8.2/nextPolish", line 515, in
main(args)
File "/apps/eb/NextPolish/1.4.0-GCCcore-8.3.0-Python-3.8.2/nextPolish", line 369, in main
task.run.start()
File "/apps/eb/NextPolish/1.4.0-GCCcore-8.3.0-Python-3.8.2/lib/python3.8/site-packages/paralleltask/task_control.py", line 344, in start
self._start()
File "/apps/eb/NextPolish/1.4.0-GCCcore-8.3.0-Python-3.8.2/lib/python3.8/site-packages/paralleltask/task_control.py", line 398, in _start
self.submit(job)
File "/apps/eb/NextPolish/1.4.0-GCCcore-8.3.0-Python-3.8.2/lib/python3.8/site-packages/paralleltask/task_control.py", line 252, in submit
_, stdout, _ = self.run(job.cmd)
File "/apps/eb/NextPolish/1.4.0-GCCcore-8.3.0-Python-3.8.2/lib/python3.8/site-packages/paralleltask/task_control.py", line 288, in run
log.critical("Command '%s' returned non-zero exit status %d, error info: %s." % (cmd, p.returncode, stderr))
File "/apps/eb/Python/3.8.2-GCCcore-8.3.0/lib/python3.8/logging/init.py", line 1481, in critical
self._log(CRITICAL, msg, args, **kwargs)
File "/apps/eb/Python/3.8.2-GCCcore-8.3.0/lib/python3.8/logging/init.py", line 1577, in _log
self.handle(record)
File "/apps/eb/Python/3.8.2-GCCcore-8.3.0/lib/python3.8/logging/init.py", line 1587, in handle
self.callHandlers(record)
File "/apps/eb/Python/3.8.2-GCCcore-8.3.0/lib/python3.8/logging/init.py", line 1649, in callHandlers
hdlr.handle(record)
File "/apps/eb/Python/3.8.2-GCCcore-8.3.0/lib/python3.8/logging/init.py", line 950, in handle
self.emit(record)
File "/apps/eb/NextPolish/1.4.0-GCCcore-8.3.0-Python-3.8.2/lib/python3.8/site-packages/paralleltask/kit.py", line 42, in emit
raise Exception(record.msg)
Exception: Command 'qsub -pe smp 5 -l vf=2.5G -o /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/00.lgs_polish/01.db_split.sh.work/db_split1/nextPolish.sh.o -e /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/00.lgs_polish/01.db_split.sh.work/db_split1/nextPolish.sh.e /scratch/kcl58759/Eco_pacbio_kendall/Nextpolish_dir/00.lgs_polish/01.db_split.sh.work/db_split1/nextPolish.sh' returned non-zero exit status 1, error info: .
Operating system
I am running on UGA's cluster system Sapelo2, I used slurm.
GCC
What version of GCC are you using?
You can use the command
gcc -vto get it.Python
What version of Python are you using?
You can use the command
python --versionto get it.NextPolish
NextPolish/1.4.0-GCCcore-8.3.0-Python-3.8.2
The text was updated successfully, but these errors were encountered: