sqm_reads2tables.py problem

[fpeti11]

I was running sqm_reads with nice results. To have a better format result I tried to use sqm_reads2tables.py.
I managed to run sqm_reads2tables.py for some sample, but most of them stopped after ~10-15 minutes with only a killed massage.

I used the script this way:
sqm_reads2tables.py B1_sqm B1_table

Can you give me an advice how to run this script?

Thank you in advance
Peter

[fpusan]

You most likely ran out of memory, where are you running this?

[fpeti11]

I was running on a HPC server with ~16GB RAM.
Then I try to allocate more memory.

[fpusan]

Yes, let me know if this helps!

[fpeti11]

I allocated ~50BG RAM.
But I got the following error.

Traceback (most recent call last):
File "/home/vad_ppm/anaconda3/envs/SqueezeMeta152/bin/sqmreads2tables.py", line 397, in
main(parse_args())
File "/home/vad_ppm/anaconda3/envs/SqueezeMeta152/bin/sqmreads2tables.py", line 276, in main
fun2tax[read] = tread
MemoryError

Should I allocate more memory?

[fpusan]

It would seem so. How many reads do you have?

[fpeti11]

I checked mappingstat.
Our samples are between 20 -25 millions reads.

[fpusan]

And how many samples do you have?

[fpeti11]

Total I have 50 samples. But I am running sqm_reads2tables.py for only one sample.

[fpusan]

I was double-checking the code and it shouldn't matter anyways... I'm a bit surprised that it is taking so much memory, but try increasing it even more, see if it helps.

[fpeti11]

After using a huge amount of memory, I managed to run the script. But for 2 samples I still got a killed message.

[fpusan]

How much memory was that? Were the samples that failed the ones with the most reads?

[fpeti11]

I used ~120GB RAM the same amount as I used for sqm_reads with success. I have problems with sample 25M and 27M reads. But as I saw we had samples with more reads that worked fine.

[fpusan]

Just realized this opened issue went under my radar. Feel free to add any update you have on this.
I will leave the issue open since this is an area I'd like to improve eventually, if I find the time.