Why neural architecture is not hardwired for N-dimensional vision but hardwired for abstract math? In The Theoretical Minimum, in lecture 1, Leonard Susskind says that you can only visualize 3 dimentional images. (see yourself). Therefore, he says, in order to deal with N dimensions, you need to deal with abstract algebra. I wonder why the brain is hardwired for abstract algebra. 

It might be worth to note that mnemotechnicians remind us that your mind thinks in pictures, and remembers pictures, not words or abstract thoughts?
Abstract algebra and n-dimensional space are constructs of the human mind. Since they are constructs of the human mind, it is of course possible for the human mind to grasp them.

N-dimensional space is a mathematical construct. That human mathematics allows for the construction of something, does not mean that it exists in reality.

For example, in mathematics, there are 3 apples. In reality, there are no 3 apples, because all apples are different (in size, color, ripeness, taste etc.), and the equation 1 apple = 1 apple is always false.

Mathematics is an abstraction from reality, and the fact that it is an abstraction means that relevant information is lost in mathematical representation. If you turn this around, it means that a mathematical model has an unkown relation to reality, because you don't know which information is not present in the model.

Even more basic, it is unknown if anything relating to that model even exists. For example, the equation 3 - 4 = -1 has no relation in reality. There are no negative objects or lengths. (All scales that include negatives, such as temperature, have an arbitrary zero point and can be transformed to a scale without negatives, e.g. Kelvin.) Even simple mathematics is fictional. The existence of a mathematical object or model, such as n-dimensional space, does not prove, nor even imply, the existence of a related reality.