title: Melt only highest values in matrix
body: <p>I have a large, square matrix containing correlations between each row or column:</p>
  
  <pre><code>mat &lt;- matrix(rnorm(5000 * 5000), nrow = 5000, dimnames = list(
      paste0("ID", seq_len(5000)), paste0("ID", seq_len(5000))))
  </code></pre>
  
  <p>I want to extract the top 100,000 unique pairs from this matrix (i.e., ID1—ID2 is the same as ID2—ID1), then convert them to a data frame. Currently, I am using this code:</p>
  
  <pre><code>corDat &lt;- reshape2::melt(mat, varnames = c("id.A", "id.B"),
                         value.name = "value", na.rm = T)
  corDat &lt;- corDat[as.character(corDat$id.A) &lt; as.character(corDat$id.B),]
  corDat &lt;- corDat[order(-corDat$value),]
  top_n &lt;- 100000
  corDat &lt;- corDat[seq_len(top_n),]
  </code></pre>
  
  <p>The result: </p>
  
  <pre><code>head(corDat)
  
             id.A   id.B    value
  19316931 ID1931 ID3864 5.658092
  14312231 ID2231 ID2863 5.416562
  3225529   ID529  ID646 5.357433
  3492653  ID2653  ID699 5.297154
  17046659 ID1659 ID3410 5.105343
  3323364  ID3364  ID665 4.987266    
  ...
  </code></pre>
  
  <p>However, because the matrix is large, the first two operations above (<code>melt</code>ing the matrix and removing duplicate pairs) take a very long time, often in excess of 5 minutes. I need to apply this operation to several hundred square matrices of variable size (often larger than 5000x5000). </p>
  
  <p>I am sure there must be a faster way to extract this information, because I do not actually need to melt the entire matrix—just the row and column names associated with the top 100,000 values. How can this operation be accomplished more efficiently?</p>