Warning: sparse->dense coercion: allocating vector of size 2.2 GiB

[satkinson0115]

Hello,

I'm trying to run normalization on an integrated Seurat object. I was able to run runEscape with no problem, but when I try performNormalization I'm getting a sparce->dense coercion warning/error. Here is my normalization command:

all_int <- performNormalization(all_int, assay = "escape.ssGSEA",
+                                 gene.sets = final_mouse_GO,
+                                 make.positive = TRUE)
[1] "Calculating features per cell..."
[1] "Normalizing enrichment scores per cell..."
Warning message:
In asMethod(object) :
  sparse->dense coercion: allocating vector of size 2.2 GiB

When I check my assays (head(all_int@assays)) ssGSEA_normalized is generated with the same number of features and cells as ssGSEA (not normalized), but I don't think normalization has been performed correctly. When I previously ran this dataset normalization took at least 10 hours; this warning is occurring right after the "Normalizing enrichment scores per cell" step starts and halts the process.

Output of the escape portions of head(all_int@assays):

$escape.ssGSEA
Assay (v5) data with 10213 features for 29088 cells
First 10 features:
 GOBP-10-FORMYLTETRAHYDROFOLATE-METABOLIC-PROCESS, GOBP-2-OXOGLUTARATE-METABOLIC-PROCESS,
GOBP-2FE-2S-CLUSTER-ASSEMBLY, GOBP-3-PHOSPHOADENOSINE-5-PHOSPHOSULFATE-METABOLIC-PROCESS,
GOBP-3-UTR-MEDIATED-MRNA-DESTABILIZATION, GOBP-3-UTR-MEDIATED-MRNA-STABILIZATION,
GOBP-4-HYDROXYPROLINE-METABOLIC-PROCESS, GOBP-4FE-4S-CLUSTER-ASSEMBLY,
GOBP-5-PHOSPHORIBOSE-1-DIPHOSPHATE-METABOLIC-PROCESS, GOBP-9-CIS-RETINOIC-ACID-METABOLIC-PROCESS 
Layers:
 data 

$escape.ssGSEA_normalized
Assay (v5) data with 10213 features for 29088 cells
First 10 features:
 GOBP-10-FORMYLTETRAHYDROFOLATE-METABOLIC-PROCESS, GOBP-2-OXOGLUTARATE-METABOLIC-PROCESS,
GOBP-2FE-2S-CLUSTER-ASSEMBLY, GOBP-3-PHOSPHOADENOSINE-5-PHOSPHOSULFATE-METABOLIC-PROCESS,
GOBP-3-UTR-MEDIATED-MRNA-DESTABILIZATION, GOBP-3-UTR-MEDIATED-MRNA-STABILIZATION,
GOBP-4-HYDROXYPROLINE-METABOLIC-PROCESS, GOBP-4FE-4S-CLUSTER-ASSEMBLY,
GOBP-5-PHOSPHORIBOSE-1-DIPHOSPHATE-METABOLIC-PROCESS, GOBP-9-CIS-RETINOIC-ACID-METABOLIC-PROCESS 
Layers:
 data 

I've found recommendations on Google to check/change the memory limit using memory.limit(), but it's no longer supported in my version of R.

Any thoughts about what's happening would be greatly appreciated.
Thanks!
Samantha

SessionInfo:

> sessionInfo()
R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 10 x64 (build 19045)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats4    stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] BiocParallel_1.38.0         tidyr_1.3.1                 ggplot2_3.5.1              
 [4] RColorBrewer_1.1-3          Seurat_5.1.0                SeuratObject_5.0.2         
 [7] sp_2.1-4                    scran_1.32.0                scuttle_1.14.0             
[10] SingleCellExperiment_1.26.0 SummarizedExperiment_1.34.0 Biobase_2.64.0             
[13] GenomicRanges_1.56.1        GenomeInfoDb_1.40.1         IRanges_2.38.0             
[16] S4Vectors_0.42.0            BiocGenerics_0.50.0         MatrixGenerics_1.16.0      
[19] matrixStats_1.3.0           escape_2.2.2               

loaded via a namespace (and not attached):
  [1] RcppAnnoy_0.0.22          splines_4.4.1             later_1.3.2               tibble_3.2.1             
  [5] R.oo_1.27.0               polyclip_1.10-6           graph_1.82.0              XML_3.99-0.17            
  [9] fastDummies_1.7.3         lifecycle_1.0.4           edgeR_4.2.1               globals_0.16.3           
 [13] lattice_0.22-6            MASS_7.3-60.2             ggdist_3.3.2              magrittr_2.0.3           
 [17] limma_3.60.4              plotly_4.10.4             metapod_1.12.0            httpuv_1.6.15            
 [21] sctransform_0.4.1         spam_2.10-0               spatstat.sparse_3.1-0     reticulate_1.38.0        
 [25] cowplot_1.1.3             pbapply_1.7-2             DBI_1.2.3                 abind_1.4-5              
 [29] zlibbioc_1.50.0           Rtsne_0.17                purrr_1.0.2               R.utils_2.12.3           
 [33] msigdbr_7.5.1             GenomeInfoDbData_1.2.12   ggrepel_0.9.5             irlba_2.3.5.1            
 [37] listenv_0.9.1             spatstat.utils_3.0-5      GSVA_1.52.3               goftest_1.2-3            
 [41] RSpectra_0.16-1           dqrng_0.4.1               spatstat.random_3.2-3     annotate_1.82.0          
 [45] fitdistrplus_1.1-11       parallelly_1.37.1         DelayedMatrixStats_1.26.0 leiden_0.4.3.1           
 [49] codetools_0.2-20          DelayedArray_0.30.1       tidyselect_1.2.1          UCell_2.8.0              
 [53] UCSC.utils_1.0.0          ScaledMatrix_1.12.0       spatstat.explore_3.2-7    jsonlite_1.8.8           
 [57] BiocNeighbors_1.22.0      progressr_0.14.0          ggridges_0.5.6            survival_3.6-4           
 [61] tools_4.4.1               ica_1.0-3                 Rcpp_1.0.12               glue_1.7.0               
 [65] gridExtra_2.3             SparseArray_1.4.8         distributional_0.5.0      AUCell_1.26.0            
 [69] dplyr_1.1.4               HDF5Array_1.32.0          withr_3.0.1               fastmap_1.2.0            
 [73] bluster_1.14.0            ggpointdensity_0.1.0      rhdf5filters_1.16.0       fansi_1.0.6              
 [77] digest_0.6.36             rsvd_1.0.5                R6_2.5.1                  mime_0.12                
 [81] colorspace_2.1-0          scattermore_1.2           tensor_1.5                spatstat.data_3.1-2      
 [85] RSQLite_2.3.7             R.methodsS3_1.8.2         utf8_1.2.4                generics_0.1.3           
 [89] data.table_1.15.4         httr_1.4.7                htmlwidgets_1.6.4         S4Arrays_1.4.1           
 [93] uwot_0.2.2                pkgconfig_2.0.3           gtable_0.3.5              blob_1.2.4               
 [97] lmtest_0.9-40             XVector_0.44.0            htmltools_0.5.8.1         dotCall64_1.1-1          
[101] GSEABase_1.66.0           scales_1.3.0              png_0.1-8                 SpatialExperiment_1.14.0 
[105] rstudioapi_0.16.0         rjson_0.2.23              reshape2_1.4.4            nlme_3.1-164             
[109] cachem_1.1.0              zoo_1.8-12                rhdf5_2.48.0              stringr_1.5.1            
[113] KernSmooth_2.23-24        parallel_4.4.1            miniUI_0.1.1.1            AnnotationDbi_1.66.0     
[117] pillar_1.9.0              grid_4.4.1                vctrs_0.6.5               RANN_2.6.1               
[121] promises_1.3.0            BiocSingular_1.20.0       beachmat_2.20.0           xtable_1.8-4             
[125] cluster_2.1.6             magick_2.8.5              locfit_1.5-9.10           cli_3.6.3                
[129] compiler_4.4.1            rlang_1.1.4               crayon_1.5.3              future.apply_1.11.2      
[133] plyr_1.8.9                stringi_1.8.4             viridisLite_0.4.2         deldir_2.0-4             
[137] babelgene_22.9            munsell_0.5.1             Biostrings_2.72.1         lazyeval_0.2.2           
[141] spatstat.geom_3.2-9       Matrix_1.7-0              RcppHNSW_0.6.0            patchwork_1.2.0          
[145] sparseMatrixStats_1.16.0  bit64_4.0.5               future_1.33.2             Rhdf5lib_1.26.0          
[149] statmod_1.5.0             KEGGREST_1.44.1           shiny_1.8.1.1             ROCR_1.0-11              
[153] igraph_2.0.3              memoise_2.0.1             bit_4.0.5  

[ncborcherding]

Hey @satkinson0115,

The warning is saying that the sparse matrix that the enrichment scores are stored in is being converted to a dense matrix to calculate the normalized values. This does require more memory for the calculation, but does speed up the normalization process substantially. Which is probably why you are seeing a difference when using v2.2.2.

In summary - I don't think you need to worry about it unless your system is taxed to the limit in terms of memory.

Nick