Concurrent to my research on Krylov Subspace Methods, I have the option of exploring mathematics behind HPC a step ahead or the theory of computation (hardware, OS, compilers etc.). Currently, I know both enough to just get by. For instance, I know how to derive the equations for CG and the basics of iterative methods but I am clueless about the details and more complicated stuff like Preconditioners and Convergence. Similarly, I know the basics of Finite Element Method (Weak Form, Non weak form, stuff like Codomain and Galerkin and stuff) but won't know the depths of it. On the computational front, I know how to code serially in all possible languages and can use OpenMP and MPI sufficiently well. I don't understand hardware and caching all that well.My question is: What should one concentrate on: Math or Computation? Are they inseparable in HPC? Is it recommended for one to learn about one and not the other?EDIT : I am currently majoring in Mechanical Engineering (which I regret) and have tons of courses in engineering and computation (fluids, heat transfer and so on). I will be joining graduate school for HPC this year and I want to strengthen some aspect (Math/Comp/Hybrid) before I begin my grad studies. I like math and comp equally (so the Do what you enjoy more is redundant).
Is it preferable to concentrate on studying math or computation?
parallel computing;education
I like aeismail's answer, but I'm going to provide an alternative perspective.In optimization, it's impossible to really learn the field without understanding real analysis. Even before you tackle numerical issues, you need to understand notions of convergence of sequences, because you are going to prove in classes that algorithms converge. You're going to have to understand concepts like continuity and differentiability on more than a superficial level. Consequently, real analysis is a prerequisite for courses in nonlinear programming.My thesis relates to methods for solving ordinary differential equations. Convergence issues, specifically things like if I reduce my local error tolerance, then my calculated numerical solution approaches the true solution of the equations I'm solving are again issues that require real analysis. To develop the theory for convergence issues required me (against my advisers' wishes) to take two semesters of real analysis. (It paid off with a couple manuscripts.)However, I know there are people out there who survive quite nicely in numerical methods and HPC without taking pure mathematics classes. It really depends on the niche that you want to occupy.If you want to develop new methods, then theory classes are helpful. Theory classes are also helpful for general mathematical literacy; reading math papers becomes much, much easier.If you want to apply specific numerical methods to problems, numerical methods classes are more helpful. I believe this perspective is where aeismail is coming from, and it is a situation more common for engineers. (Disclaimer: We know each other, and graduated from the same department.)As for HPC, the impression I've gotten is that experience is the best teacher. I took a parallel programming course, and it was slightly useful, but the main message of the class was to try things and see if they worked. If it's important for your thesis research, you'll get experience in HPC. If it's not, you won't, and it probably won't matter until you want to switch gears and tackle HPC problems. My thesis hasn't been especially HPC-heavy, at least in terms of what I program, so I haven't needed to pick up that set of skills.To wrap up, you should probably concentrate on getting background in issues that relate to your thesis problem, keep in mind what you think you want to do in the future, and decide what broad, general background you need to communicate with other researchers in the community you'd like to join. Your PhD is going to be one of the last opportunities for you to take classes, and if you think you do want to learn math theory (or any subject, really), learning it on your own is considerably harder without establishing some sort of basic proficiency first.