title: Need an optimized way to handle combination of entities to improve performance
body: <p>So, I am working on a feature in a web application. The problem is like this-
  I have four different entities. Let's say those are - <code>Item1</code>, <code>Item2</code>, <code>Item3</code>, <code>Item4</code>. There's two phase of the feature. Let's say the first phase is - <code>Choose entities</code>. In the first phase, User will have option to choose multiple items for each entity and for every combination from that choosing, I need to do some calculation. Then in the second phase(let's say <code>Relocate phase</code>) - based on the calculation done in the first phase, for each combination I would have to let user choose another combination where the value of the first combination would get removed to the row of the second combination.</p>
  
  <p>Here's the data model for further clarification - </p>
  
  <pre><code>EntityCombinationTable
  (
    Id
    Item1_Id,
    Item2_Id,
    Item3_Id,
    Item4_Id
  )
  
  ValuesTable
  (
    Combination_Id,
    Value
  )
  </code></pre>
  
  <p>So suppose I have following values in both values - </p>
  
  <pre><code>EntityCombinationTable
  Id -- Item1_Id -- Item2_Id -- Item3_Id  -- Item4_Id
  1     1           1           1            1
  2     1           2           1            1
  3     2           1           1            1
  4     2           2           1            1
  </code></pre>
  
  <pre><code>ValuesTable
  Combination_Id -- Value
  1                 10
  2                 0
  3                 0
  4                 20
  </code></pre>
  
  <p>So if in the first phase - I choose (1,2) for Item1, (1,2) for Item_2 and 1 for both Item_3 and Item4, then total combination would be 2*2*1*1 = 4.</p>
  
  <p>Then in the second phase, for each of the combination that has value greater than zero, I would have to let the user choose different combination where the values would get relocated.</p>
  
  <p>For example - As only combination with Id 1 and 2 has value greater than zero, only two relocation combination would need to be shown in the second dialog. So if the user choose (3,3,3,3) and (4,4,4,4) as relocation combination in the second phase, then new row will need to be inserted in
  <code>EntityCombinationTable</code> for (3,3,3,3) and (4,4,4,4). And values of (1,1,1,1) and (2,2,1,1) will be relocated respectively to rows corresponding to (3,3,3,3) and (4,4,4,4) in the <code>ValuesTable</code>.</p>
  
  <p>So the problem is - each of the entity can have items upto 100 or even more. So in worst case the total number of combinations can be 10^8 which would lead to a very heavy load in database(inserting and updating a huge number rows in the table) and also generating all the combination in the code level would require a substantial time.</p>
  
  <p>I have thought about an alternative approach to not keep the items as combination. Rather keep separate table for each entity. and then make the combination in the runtime. Which also would cause performance issue. As there's a lot more different stages where I might need the combination. So every time I would need to generate all the combinations.</p>
  
  <p>I have also thought about creating key-value pair type table, where I would keep the combination as a string. But in this approach I am not actually reducing number of rows to be inserted rather number of columns is getting reduced.</p>
  
  <p><strong>So my question is - Is there any better approach this kind of situation where I can keep track of combination and manipulate in an optimized way?</strong></p>
  
  <p><strong>Note - I am not sure if this would help or not, but a lot of the rows in the values table will probably have zero as value. So in the second phase we would need to show a lot less rows than the actual number of possible combinations</strong></p>