Segmentation fault (core dumped)

[MoinSheam]

I am using the latest version of bwa-mem2 (2.2.1) and I indexed the database VirRefSeq_and_Virsorted_Tara_10kb.fasta (available at: /iplant/home/shared/iVirus/ExampleData/BowtieBatch/Inputs. The prefix of the index is "map".

I ran the following command:

bwa-mem2 mem -t 40 map /data/gent/vo/001/gvo00125/vsc45615/NSGV/vs2/metaspades_2/nsgv.out/final-viral-combined.fa > out.sam

output:

Looking to launch executable "/data/gent/vo/001/gvo00125/vsc45615/miniconda3/envs/bwa_mem2/bin/bwa-mem2.avx2", simd = .avx2
Launching executable "/data/gent/vo/001/gvo00125/vsc45615/miniconda3/envs/bwa_mem2/bin/bwa-mem2.avx2"

Executing in AVX2 mode!!

• SA compression enabled with xfactor: 8

• Ref file: map

• Entering FMI_search

• Index file found. Loading index from map.bwt.2bit.64

• Reference seq len for bi-index = 450497633

• sentinel-index: 139749714

• Count:
  0, 1
  1, 126050491
  2, 225248817
  3, 324447143
  4, 450497633

• Reading other elements of the index from files map

• Index prefix: map

• Read 0 ALT contigs

• Done reading Index!!

• Reading reference genome..

• Binary seq file = map.0123

• Reference genome size: 450497632 bp

• Done reading reference genome !!

---

1. Memory pre-allocation for Chaining: 5573.5297 MB
2. Memory pre-allocation for BSW: 9584.6810 MB
3. Memory pre-allocation for BWT: 3092.5670 MB

---

• Threads used (compute): 40
• No. of pipeline threads: 2

Segmentation fault (core dumped)

I have also tried thread 8 and 10.

[kmivan]

I have exactly the same problem under AVX512. But it works well in version 2.0pre2.

[tolot27]

I've the same problem (different data sets) if I use 100 threads. In runs well with 50 threads. The machine has 4 TB Ram and the genome is 200 kb with about 100X coverage reads.

How can I support debugging this issue?

[behrj]

I have the same issue. I am running on an EC2 instance with 128Gb ram but it happens already after a few seconds.
Is also using the AVX2 mode. And it happens as for the precompiled binary as well as when I compile from source. Interestingly, if I run this within valgrind, I still get a segmentation fault rather than an error report from valgrind.
Running this with gdb I found this error message:

Thread 4 "bwa-mem2.avx512" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffbfe8516c0 (LWP 56160)]
0x00005555555857e4 in FMI_search::get_sa_entries_prefetch (this=0x5555555e4180, smemArray=<optimized out>, coordArray=0x7ffbfc06b040, coordCountArray=0x7ffbfe850d70, count=<optimized out>, max_occ=<optimized out>, tid=0, id_=@0x7ffbfe850d68: 0) at src/FMI_search.cpp:1286
1286                 pos_ar[id]  = pos;

I hope this is somehow helpful.

[behrj]

I had a look at the case. The memory is allocated here:

int64_t *pos_ar = (int64_t *) _mm_malloc( mem_lim * sizeof(int64_t), 64);

It would be nice to add a check after each alloc, so it dies a bit more gracefully. And mem_lim is negative in my case. So we might be running out of that smemArray for some reason. Not sure this is any helpful, or if there is actually anyone looking at these issues.

[behrj]

It seems in my case it was a variable overflow. If I change mem_lim to int64_t, then it runs. That is in file src/FMI_search.cpp line 1264. But I might be just scratching the surface here. Not sure what is being allocated here actually makes sense.

[vasimuddin]

it seems like a bug specifically spitted out by this dataset. can any one of you provide me data to reproduce the issue?