Log and Count the exception in a Single method
Is there any more cleaner way of doing the same thing? Meaning right now I am counting the exceptions in a method and then printing out the exceptions in the next line inside the catch block. 
  Is it possible to do the both of the things in the same addException method?


It depends. You could configure the logger and add a new a appender. Then your logger serves two appenders: The (probably) println and the counting.
Otherwise, you could add the log call inside the addException method (could be renamed to handleException(...) then).


  And if the flag is true to terminate the program, then terminate the program with the proper logging as well.


I highly suggest to avoid boolean flag arguments. If you have a public api, this suggestion is a must. Look at java for a lot of bad examples.
Use two methods:

private static void addException(String cause) {
  ...
}

private static void addExceptionAndTerminate(String cause) {
  addException(...);
  System.exit(1);
}


(And I do not see the point of adding the exception if you terminate anyway, but it could be for some multithreading things.)


  What could be the best way to re write addException method do this?


I would change the method signature from String, Boolean to Exception and put all exceptions inside a synchronized list.
Only if I want to get some statistics, I would start to parse them. Which makes the live easier for everyone who is using this.
At the moment, the functionality depends on the format of the string. If you or someone else, does not use the correct format, it does not work.
therefore, you sould document this at least in the javadoc. And the correct formatting is a pain, noone will like to write like this.
And I am not sure if the implementation for concurrent map access is correct.

If you just allow an exception object, noone has to care about formatting, the synchronization is a lot easier (just add to the list) and you have all the freedom in some other method how to count, parse 
and do whatever you want with this exceptions.