Title: JVM hotspot options for large graph measure calculation:garbage collection
Tags: java;garbage-collection;heap-memory
Question: As a part of my code i need to calculate some centrality measure for a graph with 70k vertices and 700k edges. For this purpose I used array and hash map data structures. Unfortunately I ran out of memory at the middle of program. What would be the best JVM Hotspot parameters for handle this situation? Here is the exception i got:

```Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.HashMap.createEntry(Unknown Source)
    at java.util.HashMap.addEntry(Unknown Source)
    at java.util.HashMap.put(Unknown Source)
```

So i change the heap size with -Xmx6g but this parameter did not solve problem. I still have same problem with heap space. 

In my program i want to calculate some measure for each node, unfortunately JVM keep information for all nodes and try to calculate it per each node. I want to know is there any way for changing JVM in a way that remove unneeded information from memory? For example my code crash after calculating measure for 1000 nodes from 70000 nodes. Is there anyway to remove information related to this 1000 nodes from memory after calculation? Memory could assign to other nodes in this way.Is this related to garbage collector?
here is my code(which is using JUNG library)

```public class FindMostCentralNodes {
    private DirectedSparseGraph<Customer, Transaction&gt; network = new DirectedSparseGraph<Customer, Transaction&gt;();
    static String dbName="SNfinal";
    private int numberofNodes=0;
    public static void main(String[] args) throws NumberFormatException, SQLException {
        FindMostCentralNodes f=new FindMostCentralNodes();
        int counter=1;
        DirectedSparseGraph<Customer, Transaction&gt; tsn=f.getTSN();
        DistanceCentralityScorer<Customer,Transaction&gt; scorer=new DistanceCentralityScorer<Customer,Transaction&gt;(tsn,false,true,true);// un-weighted
        Collection<Customer&gt; subscribers=tsn.getVertices();

        for(Customer node:subscribers){
            String sql="update Node set dist_centrality='"+scorer.getVertexScore(node)+"' where subscriber='"+node.getName()+"'";
            DatabaseManager.executeUpdate(sql,dbName);
            System.out.println("Update node centrality measures successfully!: "+counter++);
            node=null;
        }
    }
    public DirectedSparseGraph<Customer,Transaction&gt; getTSN() throws NumberFormatException, SQLException{
        network= new DirectedSparseGraph<Customer,Transaction&gt;();
        String count="select count(*) as counter from Node";
        ResultSet rscount=DatabaseManager.executeQuery(count, dbName);
        if(rscount.next()) {
            numberofNodes=rscount.getInt("counter");
        }
        Customer [] subscribers=new Customer[numberofNodes];
        String sql="select * from Node"; 
        ResultSet rs=DatabaseManager.executeQuery(sql, dbName);
        while(rs.next()){
            Customer sub=new Customer();
            sub.setName(rs.getString("subscriber"));
            network.addVertex(sub);
            subscribers[rs.getInt("nodeID")-1]=sub;
            sub=null;
        }
        String sql2="select * from TSN";
        ResultSet rs2=DatabaseManager.executeQuery(sql2, dbName);
        while(rs2.next()){
            Transaction transaction=new Transaction(Double.parseDouble(rs2.getString("weight")));
            network.addEdge( transaction, subscribers[rs2.getInt("callerNID")-1], subscribers[rs2.getInt("calleeNID")-1] );
            transaction=null;

        }
        //garbage
        rscount=null;
        rs=null;
        rs2=null;
        subscribers=null;
        return network;
    }


}
```

Comment: For these sizes, I would seriously suggest writing your own collections; HashMap especially isn't well suited for storing small objects, it has a non-trivial size overhead.

Comment: I would set it as high as you can.  You can use up to 80% of main memory as the heap.  For example, if you have 32 GB of main memory, I would try `-Xmx24g`  If you don't have much memory I suggest you buy some.  The settings for eclipse are different to the settings for your program.  You might like to decrease eclipse to give you more memory for your program.

Comment: did you have a space in the arguments `"-Xmx 4g"`?  because that parameter should have _no space_, i.e. `"-Xmx4g"`.

Comment: No without space ;) it was just misspelling.

Comment: I thought eclipse and JVM hotspot are the same. How can i change JVM ones?

Comment: I figure it out myself. 6GB heap size didnt help neither. So probably i need to change the code...

Here is the accepted answer: I solve the problem by creating a method which every time it handle whole algorithm for 500 nodes. After each run of this method, GC now can remove the whole variable, so my problem is solved.


Here is another answer: 
Try to change heap size (-Xmx) parameter
If you don't use some items in your HashMap, use HashMap.remove method. If there are no more references to these objects, they will be collected by GC..
Use Trove collections: http://trove.starlight-systems.com/overview


Comment for this answer: If you just put objects into HashMap without removing them it doesn't matter what GC algorithm do you use. If there is a reference from HashMap to these objects, they will not be collected by GC.
Comment for this answer: I used a library for this purpose so i dont access to that part of code! I tried to use different java GC like G1, my problem still exist.

Here is another answer: The garbage collector will remove any objects which are no longer reachable from live variables in your program. It will remove any such objects before giving up and throwing an ```OutOfMemoryError```. If you think too many objects are being retained in memory, then the first course of action is to let go of any objects you don't need, so that they are no longer reachable. Since you haven't shown us any code, we can't suggest any specific changes you could make.

If you trim the unnecessary objects, but still don't have enough memory, you could investigate the use of more compact ways to store data. A key technique is the use of off-heap storage; this is more work than simply using objects, but can be more efficient in terms of both space and CPU if it is done correctly. See:


http://mechanical-sympathy.blogspot.co.uk/2012/10/compact-off-heap-structurestuples-in.html
http://kotek.net/blog/3G_map
http://code.google.com/p/vanilla-java/wiki/HugeCollections


Comment for this answer: Finding 'packratted' objects involves digging deep into all the code that your program executes, often including library code. It's quite a big job. I'm afraid i don't have time to do it for your code. I barely even have time to do it for my own!

{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 3, 'CommentCount': 6, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2013:00:08/11/13', 'FavoriteCount': None, 'PostTypeId': 1, 'Tags': '<java><garbage-collection><heap-memory>', 'Title': 'JVM hotspot options for large graph measure calculation:garbage collection', 'ViewCount': 254, 'id': 18175789, 'user_AboutMe': '', 'user_AccountId': 1914239.0, 'user_DisplayName': 'Ali', 'user_Reputation': 1649.0, 'user_Views': 284.0, 'user_id': 1727204.0}}