Error Fehler in all_fits[[i]]$result : $ when running the function fit_tissue

[SergejRuff]

Hello,

I was testing tissueResolver today using a pancreas dataset with a pseudo-bulk dataset generated via the Simbu package. My goal was to benchmark tissueResolver against other methods, including MuSiC, so I used the same pancreas islet dataset.

Unfortunately, tissueResolver did not work for me. When running the fit_tissue function, I encountered the following error:

fitted_tissue <- fit_tissue(bulk_matrix, counts(sc_pan), bootstrap = TRUE,
bootstrap_nruns = 5, bootstrap_pctcells = 10, ncores = 4)
bootstrap run 1 of 5 (using 110 cells)
fitting 10 bulks...
Fehler in all_fits[[i]]$result : $ operator is invalid for atomic vectors

Steps to Reproduce

1. Import single-cell data

sc_pan_path <- "G:/tiergenomik/AG_bioinf/research/Ruff_Sergej/08_datacollection/pancreas_cells_one_condition/"
sc_pan <- readRDS(paste0(sc_pan_path, "sc_pancrease_healthy.rds"))

cell_types_to_keep <- c("delta", "alpha", "gamma", "ductal", "acinar", "beta")
sc_pan <- sc_pan[, sc_pan$cellType %in% cell_types_to_keep]
sc_pan$cellType <- factor(sc_pan$cellType, levels = cell_types_to_keep)

2. Generate pseudo-bulk dataset

library(SimBu)

# Create annotation dataframe
ann_data_scpan <- data.frame(ID = rownames(colData(sc_pan)), cell_type = sc_pan$cellType)

# Create dataset object
dataset_scpan <- dataset(annotation = ann_data_scpan, count_matrix = counts(sc_pan), name = "sc_pan_healthy")

# Simulate pseudo-bulk
pseudobulk_pan <- simulate_bulk(
  data = dataset_scpan, scenario = "mirror_db", scaling_factor = "NONE",
  BPPARAM = SnowParam((detectCores()-1), stop.on.error = FALSE), nsamples = 10
)

# Assign to separate variables
bulk_matrix <- assay(pseudobulk_pan$bulk, 1, withDimnames = TRUE)
bulk_matrix <- as.matrix(bulk_matrix)
bulk_se <- pseudobulk_pan$bulk
ground_truth_proportions <- pseudobulk_pan$cell_fractions

3. Run fit_tissue

fitted_tissue <- fit_tissue(bulk_matrix, counts(sc_pan), bootstrap = TRUE,
                            bootstrap_nruns = 5, bootstrap_pctcells = 10, ncores = 4)

Both bulk_matrix and counts(sc_pan) are matrix objects, as required by the function.

The example provided in the documentation runs successfully, but using real data does not.

I tested two other datasets, and they also failed with the same error.

Additional Issues:

The function documentation specifies that bulkRNA and single-cell data should be provided as data frames. However, when passing data frames, the function fails due to the promote_matrix function.

The sample function used in bootstrapping has replace = FALSE. Since bootstrapping typically involves sampling with replacement, shouldn't replace = TRUE be used instead?

I would appreciate any guidance on resolving this issue, as I would still like to benchmark tissueResolver. Additionally, I am willing to contribute to tissueResolver, but I am unsure if contributions are needed or welcomed.

Thank you for your time,
Sergej

[SergejRuff]

The dataset can be downloaded using the following link: https://xuranw.github.io/MuSiC/data/EMTABsce_healthy.rds

[j4cko]

Hi @SergejRuff , thanks for reporting. I hope we will be able to look into it next week and let you know. Contributions are welcome, just open a PR and we will have a look!

Thanks,
Jakob

[SergejRuff]

thanks. The error occurs mostly when I try to use multiple cores. If I keep ncores = 1 I don't get that error. However, I do get the following error when I try a real dataset and not the example data provided by the r package:

Fehler in FUN(X[[i]], ...) : ERROR: ABNORMAL_TERMINATION_IN_LNSRCH
Ruft auf: -> fit_tissue_noboot -> mclapply -> lapply -> FUN
Ausführung angehalten

[PaulHuettl]

Hi @SergejRuff, thank you for your clarifying question. Sorry for the delayed response.

I now ran your script with some modifications (see the comments):

library(tissueResolver)
library(SimBu)


sc_pan <- readRDS("/home/phuettl/issues/EMTABsce_healthy.rds")

cell_types_to_keep <- c("delta", "alpha", "gamma", "ductal", "acinar", "beta")
sc_pan <- sc_pan[, sc_pan$cellType %in% cell_types_to_keep]
sc_pan$cellType <- factor(sc_pan$cellType, levels = cell_types_to_keep)


# Create annotation dataframe
# NOTE: I included the na.omit in order to only have valid celltype annotiations, otherwise the simulate_bulk function failed
ann_data_scpan <- na.omit(data.frame(ID = rownames(colData(sc_pan)), cell_type = sc_pan$cellType))

# Create dataset object
# NOTE: Here we now need to filter only for valid ids, i.e. with annotation not NA
count_matrix <- counts(sc_pan)[, ann_data_scpan$ID]
dataset_scpan <- dataset(annotation = ann_data_scpan, count_matrix = count_matrix, name = "sc_pan_healthy")

# Simulate pseudo-bulk
pseudobulk_pan <- simulate_bulk(
    data = dataset_scpan, scenario = "mirror_db", scaling_factor = "NONE",
    BPPARAM = SnowParam(40, stop.on.error = FALSE), nsamples = 10
)

# Assign to separate variables
bulk_matrix <- assay(pseudobulk_pan$bulk, 1, withDimnames = TRUE)
bulk_matrix <- as.matrix(bulk_matrix)
bulk_se <- pseudobulk_pan$bulk
ground_truth_proportions <- pseudobulk_pan$cell_fractions

# FIX: The critical parameter here is maxit as it controls the admissible amount of iterations
# that the optimizer is allowed to take before breaking. This can be of course highly dataset specific.
# Increasing the maxit from 2*1e3 to 2*1e4 works for this dataset and the L-BFGS-B optimizer converges
# From a short glance at fitted_tissue$log_stores it seems that tissueResolver needs around 5k to 10k iterations
# to converge
fitted_tissue <- fit_tissue(bulk_matrix, count_matrix,
    bootstrap = TRUE,
    bootstrap_nruns = 5, bootstrap_pctcells = 10, ncores = 4,
    maxit = 20000
)

The main issue here is that the number of iterations needed for the optimizer to converge is very dataset specific.

We increased the default value of maxit from 2e3 to 2e4 now.

Concerning your other issues:

• We corrected the sampling in bootstrapping to be with replacement
• We updated the documentation for the fit_tissue function with the corrected data formats for bulkdata and sclibrary

Thank you again for your valuable comments. If there are any further issues just let us know and feel free to open pull requests.

Best,
Paul