Title: Using big unicode signs in java
Tags: unicode;character-encoding;integer;compression;character
Question: I'm writing a programm for school which should compress text. So at first I want to build a kind of dictionary from a huge number of texts for compressing later.
My idea was that if i have 2 signs, I want to replace it with only 1. So at first i am building a treemap with all the pairs I have in my String.
So for example: String s = "Hello";
He -&gt; 1
el -&gt; 1
ll -&gt; 1
lo -&gt; 1
at the end my Treemap values are different high, and at a given point i want to write a rule in my dictionary. For example:
He -&gt; x
el -&gt; y
lo -&gt; z
So here is the point. I want to start with the "new signs" at the unicode number 65536 and want to increase it for every rule by 1.
When i want to reanalyze my text to pairs i think i got a error but i am not sure about this..
TreeMap<String, Integer&gt; map = new TreeMap<String, Integer&gt;();
char[] text = s.toCharArray();
String signPair = "";
// search sign in map
for (int i = 0; i < s.length()-1; i++) {
// 1.Zeichen prüfen ob &gt;65535 -&gt;2chars
if (Character.codePointAt(text, i) &gt; 65535) {
// 2.sign checking &gt;65535 -&gt;2chars
if (Character.codePointAt(text, i + 2) &gt; 65535) {
signPair = s.substring(i, i + 4);
// compensate additional chars
i += 2;
// if not there
if (!map.containsKey(signPair)) {
// Key anlegen, Value auf 1 setzen
map.put(signPair, 1);
} else {
// Key vorhanden -&gt; Value um 1 erhöhen
int value = map.get(signPair);
value++;
map.put(signPair, value);
}
At the end when i want to print my map in the console i only got � signs with a second one.. or later i also have a lot of -typ signs which i cant interpret. In my output text there are mostly signs between 5000 and 60000. No one is higher than 65535...
Is it wrong to look at the chars and substring like them or is it a mistake to get the codepoint at them?
Thanks for help!



{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 0, 'CommentCount': 0, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2016:00:05/29/16', 'FavoriteCount': None, 'PostTypeId': 1, 'Tags': '<unicode><character-encoding><integer><compression><character>', 'Title': 'Using big unicode signs in java', 'ViewCount': 57, 'id': 37511852, 'user_AboutMe': '', 'user_AccountId': 8535219.0, 'user_DisplayName': 'NoIdea123', 'user_Reputation': 131.0, 'user_Views': 26.0, 'user_id': 6397318.0}}