Removing and counting words that appear more than once
Your algorithm seems fine. It doesn't reveal any special tricks to me, but I also don't see any unnecessary steps. I think I'd have written it myself that way. The only part that I might have done differently is the regex; I'm not sure that one is fully correct.

So on an algorithmic level you're doing perfectly fine. However, the implementation suffers from spelling issues.

Comments

First, you have some comments.

    // replace the special characters
    String freeString = longString.replaceAll("[^\\w\\s-]", "");
    //the new string is  
    System.out.println("freeString ::  " + freeString);
    //Calculate the length of the string
    String[] countOfString = freeString.split(" ");


Comments are nice, they help understand what's going on... but so does the code. Of course replaceAll replaces things, we already know that the new string is the freeString, and that String.split calculates the length of the string is actually not what it does.

You should replace your comments with comments with non-obvious things about what the code does (What does your regex actually do?), or comments about why the code does.

Variable Naming

Variable naming should be based on what the variable stores - usually its "semantic type". A "syntactic type" would be a String, Integer and so on. The "semantic type" is what it means - "inputString", "validatedString", "splittedString", "stringsToLoopThrough"... some of these are better than others. A semantic type doesn't have to include the syntactic type - you can have variable names like "index", "key", "value", without containing any notion of syntactic type. Usually the syntactic type is inferred, and at other times, it doesn't matter what the type is, just that it's the index/key/value/whatever.

Based on that, I think you should name countOfString words, uniqueString uniqueWords, countMap wordCountMap, and countString cleanedLongString.

countOfString seems to indicate a number. But it's a string array instead. A string array containing the words of the longString. So words seems like a fitting name. It certainly seems logical that if you split a longString on a space, you will get words.

uniqueString to uniqueWords is because of two reasons. First, by changing to Words, we signal that this variable is related to the words variable. Second, by pluralizing the variable name from String to Strings (or Word -> Words), we say this is a "collection" of sorts. Something which holds multiple values.

countMap to wordCountMap is to signal the relationship with words.

countString to cleanedLongString has two reasons, first to again note a relationship with another variable, this time with longString. Second, I add the cleaned because you have cleaned the special characters out of the original string. 

The names aren't ideal yet - longString seems like a bad name, considering it doesn't have to be a longString, and it doesn't signal that it needs to be cleaned first.