Optimizing Sieve of Eratosthenes for larger numbers
Your question implies that you have a sieve of Erastosthenes, but you do not.

The sieve works from known-primes forward to a value with a given magnitude.

In other words, you start with a known prime (2), and then you eliminate all the multiples of 2 until you get to the end-point of the sieve. Then, you move forward from that known prime, to the next prime (which is the next number in your sieve that is prime). You then eliminate all multiples of that next prime too.

This ends up looking like a 2-stage process with a setup stage, and a double-loop stage.

Your code has a setup stage that is not used, and then has a three-loop structure that does things back-to-front.

The setup stage creates the sieve, which is a fixed size, that needs to 'cover' the 'nth' prime, wherever that may be. You may have to cover more than the exact value, which is why the formula 2*nthPrime*(int)Math.log(nthPrime) looks handy, but you should investigate this Wikipedia article about the approximate value of the Nth prime

So, once you have a large enough 'sieve', you then loop through it as follows:

for (int possiblePrime = 2; i &lt; sieve.size; possiblePrime++) {
    if (sieve[possiblePrime]) {
        for (int notPrime = possiblePrime * 2; notPrime &lt; sieve.size; notPrime += possiblePrime) {
            sieve[notPrime] = false;
        }
    }
}


Now, using those loops, you have a basic sieve of Eratosthenes, where all true values left in the sieve are primes. If the sieve is large enough, and you count them as you find them, then you can pull the Nth prime easily.

Note that the basic algorithm can be optimized in a bunch of ways.