Bring phone numbers in consistent format
There are two aspects to this, the general design, and the implementation.
Implementation
The entire things should be extracted as a function. The first line of code:

String number = allContactNumbers.get(i).get(j);


indicates that this code is being run inside a loop (i,j). You need to extract it to be:
String number = normalizePhoneNumber(allContactNumbers.get(i).get(j));

and the function would look something like:
public String normalizePhoneNumber(String number) {
    ......
    return normalized;
}

Right, about that function, putting your code in it ends up with:
public String normalizePhoneNumber(String number) {

    number = number.replaceAll(&quot;[^+0-9]&quot;, &quot;&quot;); // All weird characters such as /, -, ...
    
    String country_code = getResources().getString(R.string.countrycode_de);
    
    if (number.substring(0, 1).compareTo(&quot;0&quot;) == 0 &amp;&amp; number.substring(1, 2).compareTo(&quot;0&quot;) != 0) {
        number = &quot;+&quot; + country_code + number.substring(1); // e.g. 0172 12 34 567 -&gt; + (country_code) 172 12 34 567
    }
    
    number = number.replaceAll(&quot;^[0]{1,4}&quot;, &quot;+&quot;); // e.g. 004912345678 -&gt; +4912345678

    return number;
}

Design
OK, about the design.... I believe this is a problem. Handling phone numbers is much more complicated than what you have.... it is really a challenging problem.
It is easy enough to strip off the junk, but it gets hard really fast. For example, this is the the Queen of England's land-line number (not kidding):

Public Information Officer
Buckingham Palace
London SW1A 1AA
Tel (during 9am - 5pm (GMT) Monday to Friday): (+44) (0)20 7930 4832. Please note, calls to this number may be recorded.


As a Canadian, the international dialing code for this would be:
01144207934832

How will your script translate that in to:
+442079305832

I think you need to revise your plan for this problem, it is more complicated than you realize.