What causes an individual to not be able to hold all the details of a large project in memory? Background: First of all, I'm not in any kind related to cognitive science, I'm a programmer. I'm not even sure that it's the right place for asking my question, but I'll try anyway. So, first of all, I'll describe the initial problem that motivated me to ask this question. There are some concrete examples.


I join the programmers team that have rather big (but not huge) project. I'm trying to figure out how it works, how each part of the system cooperates with each other. I want to see how it works as a whole, so that I could say that I understand what's going on: what goes first, what goes next, without going into pretty much detail. I want to have this image in mind and to discuss the project with no need to diving into code, or to schemes describing the project, or any additional info. The thing that does not let me do this -- I fail to remember the system components, their execution precedence that may vary, and their interactions between each other.
I got explanations of some project use case. Everything seems to be clear, but there comes a time when I realize that I can not hold all this in my head, can not keep all precedence of actions, the interactions between them. The whole thing becomes blurry.


I tend to see these cases having the same reason. As this reason results in my wish for better results achievement, I call it a problem. It may or may not be a "problem" from medical or any other point of view, but this is a problem for me anyway. So, when I talk about solving this problem, I talk about improving my results in area described above.

Questions


What is this problem called where a person can not hold all the details of a large project in memory?
Is this a failure of memory?
Is the problem due to lack of intelligence? 
Would any of the following reduce the problem: chess, poker, solving math problems, nootropics?
I just completed an extensive study (and corresponding lit review) of how people learn.  I think you are running into the classic difference between expert and novice users.

Experts vs Novices

It has been demonstrated that the knowledge structures in experts are different than those in novices.  In particular, experts' knowledge is far more structured, and they are capable of immediately connecting concepts that novices are unable to see.  (I have articles I can cite if you wish, but I'm just trying to get you on the right track here).

Bascially, people learn by slowly (or quickly) becoming "experts," that is, by organizing their conceptual models better around various concepts, such that the distinctions between them disappear into this web of schematic knowledge.

It has also been observed that novices remember learning "episodes" - e.g. hands-on exploration activities or (sometimes) lectures.  As the novice becomes an expert, the details of these individual episodes tend to fuzz together and a unified cognitive schema emerges.

Short-term Working Memory

As was mentioned in a previous answer, short-term memory can hold 5-9 discreet "chunks" of information.  To use programming terms, a chunk can either be a struct, where it represents actual data stored in working memory, or it can be a pointer to the location in long-term memory where a concept is stored.  This helps explain why experts can perform so much more quickly than novices, because their working memory can be much more efficiently utilized.

People rely on working memory to learn. As you learn, new information is incorporated into your existing mental model (this is known as constructivism).  This is an active process requiring the information to be reprocessed in working memory before it is stored into LTM.

Implications

The research above suggests that you need to take the system in pieces.  Learn what each piece does via a series of hands-on episodes, then move on to the next piece.  Then, come back and re-start this process.  As you do so, you will find yourself becoming more and more familiar with this complex system.