Combined database for overall classification in one run

[gilmahu]

Good morning,
I am new to Metabuli and finds it interesting. But on the first try i realized that it has separate databases.
Is there a way to combine all the pre-build databases into one in order to have a full clasification in a single run?
i.e. virus + bacteria + fungi + archea all in one like kraken databases

[jaebeom-kim]

Thank you for reaching out :)
I'm sorry that we don't support it now. It sounds a useful feature though. We will consider it for later updates!
But you can update the gtdb+virus+human database with your sequence of interest.

[gilmahu]

Thank you. In a previous issue i was also wondering what are the downstream compatible tools for relative abundance estimation like what bracken does when we use kraken.

[gilmahu]

@jaebeom-kim For practical reasons, it is important to have a combined database for bacteria fungi and viruses, human all in one to alow for full classification and downstream use. I tried to build a combined database following your description on the home page. i had a script with all these in place below but after 20 days running on my HPC i did not get any result and the error log was also empty and ultimately it run out of time and stopped. How do i build a metabuli db to work well?
metabuli createnewtaxalist
/data/hounmanouym/mag_to_kraken_db/metabului_db/gtdb+virus+human
/data/hounmanouym/mag_to_kraken_db/metabului_db/euk_fasta_list.txt
/data/hounmanouym/mag_to_kraken_db/metabului_db/ncbi_tax
/data/hounmanouym/mag_to_kraken_db/metabului_db/ncbi_tax/all.accession2taxid
/data/hounmanouym/mag_to_kraken_db/metabului_db/euk_taxa_input_v2

[jaebeom-kim]

I will try to solve this problem as soon as possible.
Could you please share /data/hounmanouym/mag_to_kraken_db/metabului_db/euk_fasta_list.txt ?
I want to also know how you prepared /data/hounmanouym/mag_to_kraken_db/metabului_db/ncbi_tax/all.accession2taxid and size of this file.
At last, I want to confirm if /data/hounmanouym/mag_to_kraken_db/metabului_db/ncbi_tax is downloaded from NCBI.

[gilmahu]

Hi @jaebeom-kim ,

1. The euk_fasta_list.txt is made of 20648 genomes downloaded with ncbi-genome-download and a snippet of the content is as follows: with full paths
   /data/Segrelab/GildasNH/genbank/fungi/fastas/GCA_013624395.1_ASM1362439v1_genomic.fna
   /data/Segrelab/GildasNH/genbank/fungi/fastas/GCA_013694445.1_ASM1369444v1_genomic.fna
   /data/Segrelab/GildasNH/genbank/fungi/fastas/GCA_013753835.1_ASM1375383v1_genomic.fna
   etc..

2. everything in ncbi_tax were downloaded from NCBI directly and unzipped as follows:

wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/accession2taxid/nucl_gb.accession2taxid.gz
wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/accession2taxid/nucl_wgs.accession2taxid.gz
wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/accession2taxid/wgs.accession2taxid.gz

Then:
gunzip *.gz
cat nucl_gb.accession2taxid nucl_wgs.accession2taxid wgs.accession2taxid > /data/metabuli/ncbi_tax/all.accession2taxid

Ultimately, inside the ncbi_tax this is the content below:

[hounmanouym@cn4270 kraken_db_gil]$ ls -lh ../metabului_db/ncbi_tax/
total 89G
-rw-r----- 1 hounmanouym hounmanouym 76G May 13 17:46 all.accession2taxid
drwxr-x--- 2 hounmanouym hounmanouym 4.0K May 13 17:37 all_map
-rw-r----- 1 hounmanouym hounmanouym 19M May 12 11:28 citations.dmp
-rw-r----- 1 hounmanouym hounmanouym 6.2M May 12 11:25 delnodes.dmp
-rw-r----- 1 hounmanouym hounmanouym 452 May 12 11:20 division.dmp
-rw-r----- 1 hounmanouym hounmanouym 17K May 12 11:28 gc.prt
-rw-r----- 1 hounmanouym hounmanouym 4.9K May 12 11:20 gencode.dmp
-rw-r----- 1 hounmanouym hounmanouym 4.6M May 12 11:25 images.dmp
-rw-r----- 1 hounmanouym hounmanouym 1.7M May 12 11:25 merged.dmp
-rw-r----- 1 hounmanouym hounmanouym 255M May 12 11:28 names.dmp
-rw-r----- 1 hounmanouym hounmanouym 193M May 12 11:27 nodes.dmp
-rw-r----- 1 hounmanouym hounmanouym 13G May 10 04:35 nucl_gb.accession2taxid
-rw-r----- 1 hounmanouym hounmanouym 3.1K Mar 26 15:16 readme.txt

[gilmahu]

@jaebeom-kim did you get to do anything on this since last time?

[jaebeom-kim]

Hi! Sorry for this delayed response.
I think it is just too slow.
20,648 genomes are large, and all.accession2taxid is also large.
createnewtaxalist command is not parallelized.
I will try to make it faster.

Could you provide euk_fasta_list.txt, so that I can download them and reproduce this issue?

[jaebeom-kim]

Hi. I hope you doing well.
Could you please provide euk_fasta_list.txt if possible.
Having it would help me tackle the problem.