Is it poor grammar to replace normal phrases with mathematical symbols in sentences in a mathematical paper?
Are expressions such as "The matrix A has rank ≥ n" considered as acceptable in mathematical papers/theses/textbooks?


No, it's often considered poor style to incorporate fragments of equations like this into text.  I wouldn't go so far as to say it's ungrammatical, but many people consider it bad writing.  Some others don't care about this issue, which is why you sometimes see it done, but this is more common in informal or unedited writing.

The issue is that "rank ≥ n" is mixing together English and mathematics within the same construction.  If this doesn't bother you, imagine a more dramatic case like "n + five".  (By contrast, when someone writes "if x ≥ y", the inequality "x ≥ y" is a self-contained unit within the sentence.)  There's no logical reason why mathematical writing conventions couldn't allow this sort of mixing, but they don't.

Saying "The matrix A has rank at least n" is shorter and cleaner than "The rank of the matrix A is greater than or equal to n", but they are both acceptable.  I'd recommend avoiding "The matrix A has rank ≥ n" (I can't think of a good reason to prefer it, and avoiding looking bad is a reason not to use it).