Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

empty Reportfile #143

Open
JochenSchaefergmxde opened this issue May 28, 2023 · 6 comments
Open

empty Reportfile #143

JochenSchaefergmxde opened this issue May 28, 2023 · 6 comments

Comments

@JochenSchaefergmxde
Copy link

JochenSchaefergmxde commented May 28, 2023

krakenuniq -db /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb --threads 32 --exact --output /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_O.txt --report-file /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_R.txt /mnt/fastq/rz_TSAA8162_t2t_u_o.fastq.gz
Warning: Overwriting /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_R.txt.
/usr/local/bin/classifyExact -d /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb/database.kdb -i /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb/database.idx -t 32 -o /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_O.txt -r /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_R.txt -a /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb/taxDB -p 12
Database /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb/database.kdb
Loaded database with 34335766180 keys with k of 31 [val_len 4, key_len 8].
Reading taxonomy index from /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb/taxDB. Done.
Writing Kraken output to /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_O.txt
59137302 sequences (7550.96 Mbp) processed in 3559.795s (996.8 Kseq/m, 127.27 Mbp/m).
58871764 sequences classified (99.55%)
265538 sequences unclassified (0.45%)
Writing report file to /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_R.txt ..
Reading genome sizes from /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb/database.kdb.counts ... done
Setting values in the taxonomy tree ...(base)

Whats the Problem ??

cat /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_R.txt

#KrakenUniq v1.0.4 DATE:2023-05-28T09:36:43Z DB:/mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb DB_SIZE:412029195232 WD:/mnt/sdc1/rz/reports
#CL:/usr/local/bin/krakenuniq -db /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb --threads 32 --exact --output /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_O.txt --report-file /mnt/sdc1/rz/2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_R.txt /mnt/fastq/rz_TSAA8162_t2t_u_o.fastq.gz

and nothing more, but in the Outputfile there are about 3792 Taxids listet.
cut -f3 2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_O.txt|sort|uniq -c|sort -nr > 2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_O.csv

wc 2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_O.csv
3792 7584 56960 2023_05_28_10_24_10_rz_kuniq_stanTSAA8162t2tu_O.csv

So what could be wrong ? The System has a free -h
gesamt benutzt frei gemns. Puffer/Cache verfügbar
Speicher: 125Gi 2,0Gi 118Gi 64Mi 6,1Gi 123Gi
Auslager: 136Gi 420Mi 136Gi

The Reportfile is also Null if i use the preload 8GB switch (--preload-size 8GB). !?

I try to convert the FastQ-File with fastp but nothing changed .

@JochenSchaefergmxde
Copy link
Author

2023-06-11T06:03:41.766209+02:00 linux systemd[1]: fwupd.service: Deactivated successfully.
2023-06-11T06:08:54.189003+02:00 linux kernel: [66989.977652] classifyExact invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=0, oom_score_adj=0
2023-06-11T06:08:54.208257+02:00 linux kernel: [66989.977659] CPU: 23 PID: 6341 Comm: classifyExact Not tainted 6.2.0-20-generic #20-Ubuntu
2023-06-11T06:08:54.208259+02:00 linux kernel: [66989.977661] Hardware name: Gigabyte Technology Co., Ltd. Z790 GAMING X AX/Z790 GAMING X AX, BIOS F3k 01/05/2023
2023-06-11T06:08:54.208259+02:00 linux kernel: [66989.977662] Call Trace:
2023-06-11T06:08:54.208259+02:00 linux kernel: [66989.977664]
2023-06-11T06:08:54.208260+02:00 linux kernel: [66989.977666] dump_stack_lvl+0x48/0x70
2023-06-11T06:08:54.210425+02:00 linux kernel: [66989.977672] dump_stack+0x10/0x20
2023-06-11T06:08:54.210526+02:00 linux kernel: [66989.977674] dump_header+0x50/0x290
2023-06-11T06:08:54.210528+02:00 linux kernel: [66989.977677] oom_kill_process+0x10d/0x1c0
2023-06-11T06:08:54.210528+02:00 linux kernel: [66989.977678] out_of_memory+0x101/0x370
2023-06-11T06:08:54.210528+02:00 linux kernel: [66989.977679] __alloc_pages_may_oom+0x112/0x1e0
2023-06-11T06:08:54.210528+02:00 linux kernel: [66989.977682] __alloc_pages_slowpath.constprop.0+0x4c7/0xa20
2023-06-11T06:08:54.210529+02:00 linux kernel: [66989.977684] __alloc_pages+0x31d/0x350
2023-06-11T06:08:54.210529+02:00 linux kernel: [66989.977686] __folio_alloc+0x1d/0x60
2023-06-11T06:08:54.210530+02:00 linux kernel: [66989.977687] ? policy_node+0x69/0x80
2023-06-11T06:08:54.210530+02:00 linux kernel: [66989.977690] vma_alloc_folio+0x9f/0x3d0
2023-06-11T06:08:54.210530+02:00 linux kernel: [66989.977691] __read_swap_cache_async+0x119/0x290
2023-06-11T06:08:54.210530+02:00 linux kernel: [66989.977694] swap_cluster_readahead+0x250/0x3a0
2023-06-11T06:08:54.210530+02:00 linux kernel: [66989.977696] ? __mod_memcg_lruvec_state+0x67/0x100
2023-06-11T06:08:54.212540+02:00 linux kernel: [66989.977698] swapin_readahead+0x45/0x80
2023-06-11T06:08:54.212547+02:00 linux kernel: [66989.977700] do_swap_page+0x185/0x930
2023-06-11T06:08:54.212547+02:00 linux kernel: [66989.977702] handle_pte_fault+0x202/0x240
2023-06-11T06:08:54.212547+02:00 linux kernel: [66989.977704] __handle_mm_fault+0x661/0x720
2023-06-11T06:08:54.212547+02:00 linux kernel: [66989.977706] handle_mm_fault+0x119/0x330
2023-06-11T06:08:54.212548+02:00 linux kernel: [66989.977708] do_user_addr_fault+0x1c1/0x720
2023-06-11T06:08:54.212550+02:00 linux kernel: [66989.977712] exc_page_fault+0x80/0x1b0
2023-06-11T06:08:54.212550+02:00 linux kernel: [66989.977714] asm_exc_page_fault+0x27/0x30
2023-06-11T06:08:54.212551+02:00 linux kernel: [66989.977717] RIP: 0033:0x7fcab9371330
2023-06-11T06:08:54.212551+02:00 linux kernel: [66989.977720] Code: c5 fe 6f 56 40 c5 fe 6f 5e 60 c5 fe 6f a6 00 10 00 00 c5 fe 6f ae 20 10 00 00 c5 fe 6f b6 40 10 00 00 c5 fe 6f be 60 10 00 00 7e 6f 86 00 20 00 00 c5 7e 6f 8e 20 20 00 00 c5 7e 6f 96 40 20
2023-06-11T06:08:54.212552+02:00 linux kernel: [66989.977721] RSP: 002b:00007f68c0d9bcf8 EFLAGS: 00010206
2023-06-11T06:08:54.212552+02:00 linux kernel: [66989.977723] RAX: 00007f2367fe6010 RBX: 00007f68c0d9bd60 RCX: 0000000000000020
2023-06-11T06:08:54.212552+02:00 linux kernel: [66989.977723] RDX: 0000000000003fd0 RSI: 00007f61c4e11040 RDI: 00007f25c4e02040
2023-06-11T06:08:54.212552+02:00 linux kernel: [66989.977724] RBP: 00007f68c0d9bd90 R08: ffffffffffffffd0 R09: 0000000000000000
2023-06-11T06:08:54.212552+02:00 linux kernel: [66989.977725] R10: 0000000000068c78 R11: 0000000001980000 R12: 0000000080000000
2023-06-11T06:08:54.212553+02:00 linux kernel: [66989.977726] R13: 00007f68842f1040 R14: 0000000400000000 R15: 00005634616a78e0
2023-06-11T06:08:54.212553+02:00 linux kernel: [66989.977728]
2023-06-11T06:08:54.212626+02:00 linux kernel: [66989.977729] Mem-Info:
2023-06-11T06:08:54.212631+02:00 linux kernel: [66989.977730] active_anon:711489 inactive_anon:31194802 isolated_anon:291
2023-06-11T06:08:54.212631+02:00 linux kernel: [66989.977730] active_file:1378 inactive_file:819 isolated_file:0
2023-06-11T06:08:54.212631+02:00 linux kernel: [66989.977730] unevictable:20280 dirty:0 writeback:57
2023-06-11T06:08:54.212632+02:00 linux kernel: [66989.977730] slab_reclaimable:373762 slab_unreclaimable:52201
2023-06-11T06:08:54.212632+02:00 linux kernel: [66989.977730] mapped:2042 shmem:17757 pagetables:335933
2023-06-11T06:08:54.212632+02:00 linux kernel: [66989.977730] sec_pagetables:0 bounce:0
2023-06-11T06:08:54.212632+02:00 linux kernel: [66989.977730] kernel_misc_reclaimable:0
2023-06-11T06:08:54.212633+02:00 linux kernel: [66989.977730] free:147488 free_pcp:0 free_cma:0
2023-06-11T06:08:54.212633+02:00 linux kernel: [66989.977733] Node 0 active_anon:2845956kB inactive_anon:124779208kB active_file:5512kB inactive_file:3276kB unevictable:81120kB isolated(anon):1164kB isolated(file):0kB mapped:8168kB dirty:0kB writeback:228kB shmem:71028kB shmem_thp: 47104kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:16544kB pagetables:1343732kB sec_pagetables:0kB all_unreclaimable? yes
2023-06-11T06:08:54.212635+02:00 linux kernel: [66989.977736] Node 0 DMA free:11264kB boost:0kB min:4kB low:16kB high:28kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
2023-06-11T06:08:54.212768+02:00 linux kernel: [66989.977739] lowmem_reserve[]: 0 641 128466 128466 128466
2023-06-11T06:08:54.212774+02:00 linux kernel: [66989.977741] Node 0 DMA32 free:511636kB boost:0kB min:336kB low:992kB high:1648kB reserved_highatomic:0KB active_anon:2176kB inactive_anon:199764kB active_file:0kB inactive_file:24kB unevictable:856kB writepending:0kB present:798416kB managed:732516kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB

@JochenSchaefergmxde
Copy link
Author

krakenuniq -db /mnt/m2/kuniqdb/kuniq_standard_plus_eupath_minus_kdb --threads 32 --exact --output /mnt/sdc1/rz/2023_06_11_11_53_54_rz_kuniq_stanTSAA8162t2tu_O.txt --preload-size 40GB --report-file /mnt/sdc1/rz/2023_06_11_11_53_54_rz_kuniq_stanTSAA8162t2tu_R.txt /mnt/fastq/rz_TSAA8162_t2t_u_o.fastq.gz

Even if i use preload-size 40GB (PC has 128GB) i get

2023-06-11T13:17:12.819421+02:00 linux kernel: [92687.959460] CPU: 19 PID: 9218 Comm: classifyExact Not tainted 6.2.0-20-generic #20-Ubuntu
2023-06-11T13:17:12.819427+02:00 linux kernel: [92687.959462] Hardware name: Gigabyte Technology Co., Ltd. Z790 GAMING X AX/Z790 GAMING X AX, BIOS F3k 01/05/2023
2023-06-11T13:17:12.819427+02:00 linux kernel: [92687.959463] Call Trace:
2023-06-11T13:17:12.819429+02:00 linux kernel: [92687.959464]
2023-06-11T13:17:12.819430+02:00 linux kernel: [92687.959466] dump_stack_lvl+0x48/0x70
2023-06-11T13:17:12.819430+02:00 linux kernel: [92687.959471] dump_stack+0x10/0x20
2023-06-11T13:17:12.819431+02:00 linux kernel: [92687.959473] dump_header+0x50/0x290
2023-06-11T13:17:12.819431+02:00 linux kernel: [92687.959476] oom_kill_process+0x10d/0x1c0
2023-06-11T13:17:12.819431+02:00 linux kernel: [92687.959477] out_of_memory+0x101/0x370
2023-06-11T13:17:12.819432+02:00 linux kernel: [92687.959479] __alloc_pages_may_oom+0x112/0x1e0
2023-06-11T13:17:12.819432+02:00 linux kernel: [92687.959482] __alloc_pages_slowpath.constprop.0+0x4c7/0xa20
2023-06-11T13:17:12.819432+02:00 linux kernel: [92687.959484] __alloc_pages+0x31d/0x350
2023-06-11T13:17:12.819433+02:00 linux kernel: [92687.959486] __folio_alloc+0x1d/0x60
2023-06-11T13:17:12.819433+02:00 linux kernel: [92687.959487] ? policy_node+0x69/0x80
2023-06-11T13:17:12.819433+02:00 linux kernel: [92687.959490] vma_alloc_folio+0x9f/0x3d0
2023-06-11T13:17:12.819434+02:00 linux kernel: [92687.959491] do_anonymous_page+0xfa/0x390
2023-06-11T13:17:12.819434+02:00 linux kernel: [92687.959494] handle_pte_fault+0x20f/0x240
2023-06-11T13:17:12.819435+02:00 linux kernel: [92687.959496] __handle_mm_fault+0x661/0x720
2023-06-11T13:17:12.819435+02:00 linux kernel: [92687.959498] handle_mm_fault+0x119/0x330
2023-06-11T13:17:12.819435+02:00 linux kernel: [92687.959500] do_user_addr_fault+0x1c1/0x720
2023-06-11T13:17:12.819435+02:00 linux kernel: [92687.959503] exc_page_fault+0x80/0x1b0
2023-06-11T13:17:12.819436+02:00 linux kernel: [92687.959506] asm_exc_page_fault+0x27/0x30
2023-06-11T13:17:12.819437+02:00 linux kernel: [92687.959509] RIP: 0033:0x7fbbda5713d7
2023-06-11T13:17:12.819438+02:00 linux kernel: [92687.959511] Code: 00 00 c5 7d e7 8f 20 20 00 00 c5 7d e7 97 40 20 00 00 c5 7d e7 9f 60 20 00 00 c5 7d e7 a7 00 30 00 00 c5 7d e7 af 20 30 00 00 7d e7 b7 40 30 00 00 c5 7d e7 bf 60 30 00 00 48 83 ef 80 ff c9
2023-06-11T13:17:12.819438+02:00 linux kernel: [92687.959512] RSP: 002b:00007f4fe3738cf8 EFLAGS: 00010203
2023-06-11T13:17:12.819438+02:00 linux kernel: [92687.959514] RAX: 00007f28e7fef010 RBX: 00007f4fe3738d60 RCX: 0000000000000001
2023-06-11T13:17:12.819439+02:00 linux kernel: [92687.959515] RDX: 0000000000003fd0 RSI: 00007f4a57064040 RDI: 00007f2c5705bfc0
2023-06-11T13:17:12.819439+02:00 linux kernel: [92687.959515] RBP: 00007f4fe3738d90 R08: ffffffffffffffd0 R09: 0000000000000000
2023-06-11T13:17:12.819439+02:00 linux kernel: [92687.959516] R10: 00000000000243e4 R11: 0000000001980000 R12: 0000000080000000
2023-06-11T13:17:12.819440+02:00 linux kernel: [92687.959517] R13: 000055d95dd6acd0 R14: 0000000400000000 R15: 000055d961354d60
2023-06-11T13:17:12.819440+02:00 linux kernel: [92687.959519]
2023-06-11T13:17:12.819441+02:00 linux kernel: [92687.959520] Mem-Info:
2023-06-11T13:17:12.819441+02:00 linux kernel: [92687.959521] active_anon:5382938 inactive_anon:26584760 isolated_anon:0
2023-06-11T13:17:12.819441+02:00 linux kernel: [92687.959521] active_file:529 inactive_file:1895 isolated_file:66
2023-06-11T13:17:12.819441+02:00 linux kernel: [92687.959521] unevictable:15757 dirty:0 writeback:0
2023-06-11T13:17:12.819442+02:00 linux kernel: [92687.959521] slab_reclaimable:316950 slab_unreclaimable:51095
2023-06-11T13:17:12.819442+02:00 linux kernel: [92687.959521] mapped:997 shmem:18568 pagetables:337704
2023-06-11T13:17:12.819442+02:00 linux kernel: [92687.959521] sec_pagetables:0 bounce:0
2023-06-11T13:17:12.819442+02:00 linux kernel: [92687.959521] kernel_misc_reclaimable:0
2023-06-11T13:17:12.819443+02:00 linux kernel: [92687.959521] free:147100 free_pcp:0 free_cma:0
2023-06-11T13:17:12.819443+02:00 linux kernel: [92687.959524] Node 0 active_anon:21531752kB inactive_anon:106339040kB active_file:2116kB inactive_file:7580kB unevictable:63028kB isolated(anon):0kB isolated(file):264kB mapped:3988kB dirty:0kB writeback:0kB shmem:74272kB shmem_thp: 45056kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:16576kB pagetables:1350816kB sec_pagetables:0kB all_unreclaimable? no

@JochenSchaefergmxde
Copy link
Author

if you install kraken after krakenuniq you get a false classify file, you must install krakenuniq at last !

@JochenSchaefergmxde
Copy link
Author

Doesn't work even you install new ! :( and you define TMP :( with 128GB RAM + 150GB Swap :( and preload-size 40G

@JochenSchaefergmxde
Copy link
Author

It seems to be the "exact" switch how made the Problems !. If the fastq is about 2GB i get a memory Error in syslog with classifyexact. classify should work, this is called without "-exact" switch ! :(

@JochenSchaefergmxde
Copy link
Author

Both Kraken2 and Krakenuniq are using classify but in different Versions !
software/krakenuniq/src/classify.cpp
software/krakenuniq/src/classifyExact
software/krakenuniq/src/classify
software/kraken2/src/classify.cc
software/kraken2/src/classify
software/kraken2/src/classify.o

So you cant install it in the same place like /usr/local/bin .

If you want to use both you must install it in different locations and attention to you PATH variable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant