Understanding the state vector in Quantum Computing
No, it's not correct.  It's not true in general that one entry will be 1 and all others 0; that is true for the basis vectors, but there are other states (other vectors) where that isn't true.  You can have linear combinations of the basis states, e.g., the following is a possible state:

$${1 \over \sqrt{2}} |0\rangle + {1 \over \sqrt{2}} |1\rangle.$$

It corresponds to the following vector in the vector space:

$${1 \over \sqrt{2}} (0,1) + {1 \over \sqrt{2}} (1,0) = (1/\sqrt{2},1/\sqrt{2}).$$

Because of this, you can have infinitely many possible states in this vector space.

The vector space is not $N$-dimensional.  The dimension of a vector space has a formal definition, and if you apply it, you will discover that the dimension of the vector space is $2^N$.