Where can I find the list of all tendons in the human body?
Your best bet is the Terminologia Anatomica, which is the international standard for anatomical terminology.

The 1998 edition is freely available. It lists only a few named tendons though, which is consistent with my experience as an anatomist: very few tendons are named separately from the muscles to which they are connected. Central tendon of the diaphragm, conjoint tendon, and calcaneal tendon are a few.

Otherwise, it's just "tendon of biceps brachii" (which actually has 2 tendons at one end and one at the other).

4,000 seems like a too high number to me. 2x the 640ish muscles plus some extra for multi-headed muscles and those with internal tendons or aponeurotic tendons seems like a more reasonable estimate.