Mathematics in inorganic and quantum chemistry
For better understanding of molecular symmetry you indeed need some mathematics, namely, some basics of group theory, but aside from that I doubt that you need some mathematics apart form arithmetic and calculus for general, inorganic, organic, and even physical chemistry. 

For quantum mechanics you absolutely need to familiarize yourself with the concept of complex numbers and you have to learn some basic approaches for solving differential equations. For an advanced insight into quantum chemistry you need some linear algebra, by which I mean the theory of vector spaces both finite- and infinite-dimensional, not just finite-dimensional part of it, which, in a sense, reduces to matrix algebra.

P.S. Abstract algebra is nothing but a common name for different areas of mathematics which studies different so-called algebraic structures. Say, group theory studies groups, linear algebra studies vector spaces, etc.