title: DESCEND's function 'selectGenes' takes forever to finish on sparse matrix, and also shows no error message
body: <p>I have a sparse matrix of dimension 33694*10000, which I had converted into data frame and then coerced into a matrix. I am using DESCEND  and SOUP (semisoft clustering of single-cell data) to filter out highly variable genes from the matrix and in turn, perform clustering.</p>
  <p>The function &quot;selectGenes&quot; from SOUP is taking too long to finish however earlier it took two days to finish the same task (different matrix but same dimensions). I have checked my matrix for any missing values or NAs but the matrix seems fine. Or maybe I need to look for some other kind of error in my matrix because the test data is running fine and taking the usual time to finish the run.</p>
  <p>I don't know how to troubleshoot this because I cannot see any error my screen just looks like this.</p>
  <pre><code>select.out_SP1_mat = selectGenes(SP1_mat_transposed, type=&quot;count&quot;, n.cores=25)
  
  &gt; select.out_SP1_mat = selectGenes(SP1_mat_transposed, type=&quot;count&quot;, n.cores=25)
  Removed 32820 genes that are expressed in less than 10 cells.
  Selecting from remaining 874 genes...
  SPCA selection...
  DESCEND selection...
  [1] &quot;DESCEND starts deconvolving distribution of 874 genes!&quot;
  [1] &quot;Estimating the time to finish this task ...&quot;
  </code></pre>
  <p>Here is the link to SOUP tutorial pdf and the code for function &quot;selectGenes&quot;:</p>
  <p><a href="https://github.com/lingxuez/SOUPR/blob/master/vignettes/SOUP-vignette.pdf" rel="nofollow noreferrer">https://github.com/lingxuez/SOUPR/blob/master/vignettes/SOUP-vignette.pdf</a></p>
  <p><a href="https://rdrr.io/github/lingxuez/SOUP/src/R/geneSelect.R" rel="nofollow noreferrer">https://rdrr.io/github/lingxuez/SOUP/src/R/geneSelect.R</a></p>
  <p><a href="https://rdrr.io/github/jingshuw/descend/src/R/descend.R" rel="nofollow noreferrer">https://rdrr.io/github/jingshuw/descend/src/R/descend.R</a></p>
  <p>Can anyone suggest how do I troubleshoot or solve this issue? I am running this on cluster having 64 cores and 1 Tb RAM.</p>
  <p>This is what I did before running the function mentioned:</p>
  <pre><code>library(SOUP)
  library(ggplot2)
  &gt; SP1 &lt;-read.table(file = &quot;/home/mverma/SINGLE_CELL_data/spleen/raw_gene_bc_matrices/spleen_subset_test/analysis/SP1_matrix.tsv&quot;, sep = '\t', header = TRUE)
  &gt; SP1_mat &lt;- as.matrix(x = SP1)
  &gt; dim(SP1_mat)
  [1] 33694 10000
  &gt; SP1_mat_transposed &lt;- t(SP1_mat)
  &gt; dim(SP1_mat_transposed)
  [1] 10000 33694
  
  &gt; log.expr_SP1_mat= log2(scaleRowSums(SP1_mat_transposed)*(10^6) + 1)
  &gt; dim(log.expr_SP1_mat)
  [1] 10000 33694
  
  &gt; select.out_SP1_mat = selectGenes(SP1_mat_transposed, type=&quot;count&quot;, n.cores=25)
  Removed 32820 genes that are expressed in less than 10 cells.
  Selecting from remaining 874 genes...
  SPCA selection...
  DESCEND selection...
  [1] &quot;DESCEND starts deconvolving distribution of 874 genes!&quot;
  [1] &quot;Estimating the time to finish this task ...&quot;
  </code></pre>