Why exactly does the immune system weaken with age?
We don't know.  

More precisely, we know of many, many different reasons why the immune system deteriorates with age, but we don't really know which are different measures of the same thing, which are independent factors, which factors actually cause problems and which are harmless and incidental, and so on.  

There are too many age-related phenomena to try to list. There's a nice recent review Aging and the immune system: An overview which spends time on 


Aging and the immune cell repertoire
Cell Intrinsic defects in lymphocytes
"Inflammaging" ("an overall decrease in the ability to cope with different stressors, accompanied by a progressive increase in the organism's pro-inflammatory status")


Each of these broad topics can be broken down into dozens of sub-topics, and the molecular and evolutionary mechanisms underlying each are rarely well understood.

It's a huge area of research, and it's not possible to give a single concise answer.