title: How to optimize performance when repeatedly looping over a big list of objects
body: <p>I have a simple file that contains two integer values per line (a source integer and a target integer). Each line represents a relation between two values. The file is not sorted and the actual file contains about 4 million lines. After sorting it may look like this:</p>
  
  <pre><code>sourceId;targetId   
  1;5    
  2;3   
  4;7  
  7;4  
  8;7  
  9;5 
  </code></pre>
  
  <p>My goal is to create a new object that will represent all unique related integers in a list with a unique identifier. The expected output of this example should be the following three objects:</p>
  
  <pre><code>0, [1, 5, 9]  
  1, [2, 3]  
  2, [4, 7, 8] 
  </code></pre>
  
  <p>So groupId 0 contains a group of relations (1, 5 and 9).</p>
  
  <p>Below is my current way to create a list of these objects. The list of Relation objects contains all the lines in memory. And the list of GroupedRelation should be the end result.</p>
  
  <pre><code>public class GroupedRelationBuilder {
  
      private List&lt;Relation&gt; relations;
      private List&lt;GroupedRelation&gt; groupedRelations;
      private List&lt;String&gt; ids;
      private int frameId;
  
      public void build() {
          relations = new ArrayList&lt;&gt;();
          relations.add(new Relation(1, 5));
          relations.add(new Relation(4, 7));
          relations.add(new Relation(8, 7));
          relations.add(new Relation(7, 4));
          relations.add(new Relation(9, 5));
          relations.add(new Relation(2, 3));
  
          // sort
          relations.sort(Comparator.comparing(Relation::getSource).thenComparing(Relation::getTarget));
  
          // build the groupedRelations
          groupId = 0;
          groupedRelations = new ArrayList&lt;&gt;();
          for (int i = 0; relations.size() &gt; 0;) {
              ids = new ArrayList&lt;&gt;();
              int compareSource = relations.get(i).getSource();
              int compareTarget = relations.get(i).getTarget();
              ids.add(Integer.toString(compareSource));
              ids.add(Integer.toString(compareTarget));               
              relations.remove(i);
              for (int j = 0; j &lt; relations.size(); j++) {
                  int source = relations.get(j).getSource();
                  int target = relations.get(j).getTarget();
                  if ((source == compareSource || source == compareTarget) &amp;&amp; !ids.contains(Integer.toString(target))) {
                      ids.add(Integer.toString(target));                      
                      relations.remove(j);
                      continue;
                  }
                  if ((target == compareSource || target == compareTarget) &amp;&amp; !ids.contains(Integer.toString(source))) {
                      ids.add(Integer.toString(source));                      
                      relations.remove(j);
                      continue;
                  }
              }
              if (relations.size() &gt; 0) {
                  groupedRelations.add(new GroupedRelation(groupId++, ids));
              }
          }
      }
  
      class GroupedRelation {
          private int groupId;
          private List&lt;String&gt; relatedIds;
  
          public GroupedRelation(int groupId, List&lt;String&gt; relations) {
              this.groupId = groupId;
              this.relatedIds = relations;
          }
  
          public int getGroupId() {
              return groupId;
          }
  
          public List&lt;String&gt; getRelatedIds() {
              return relatedIds;
          }
      }
  
      class Relation {
          private int source;
          private int target;
  
          public Relation(int source, int target) {
              this.source = source;
              this.target = target;
          }
  
          public int getSource() {
              return source;
          }
  
          public void setSource(int source) {
              this.source = source;
          }
  
          public int getTarget() {
              return target;
          }
  
          public void setTarget(int target) {
              this.target = target;
          }
      }
  }
  </code></pre>
  
  <p>When I run this small example program, it takes 15 seconds to create 1000 GroupedRelation objects. To create 1 million GroupedRelation it would take 250 minutes.. </p>
  
  <p>I am looking for help in optimizing my code that does get the result I want but simply takes to long.</p>
  
  <p><strong>Is it possible to optimize the iteration in such a way that the expected result is the same but the time it takes to get the expected result is reduced significantly? If this is possible, how would you go about it?</strong></p>