How to know which kind of distortion takes place due to Jahn-Teller effect
If you read and watched videos, you probably already know that the Jahn-Teller effect causes stabilisation in the case of unevenly populated orbitals. Reducing the symmetry of the system renders the orbitals no longer symmetrically equivalent thereby creating an energy difference used to the compound’s advantage.

Let’s separate two cases: 1) octahedral complexes with $\mathrm{d^4}$ and $\mathrm{d^9}$ configuration, in which the $\mathrm{e_g}$ orbitals are unevenly populated; and 2) all the other cases.

Uneven $\mathbf{e_g}$ population

In an octahedral complex with four or nine d-electrons, there will be either one or three within the $\mathrm{e_g}$ orbitals ($\mathrm{d}_{x^2-y^2}$ and $\mathrm{d}_{z^2}$). It is unfavourable for orbitals of the same energy (as symmetrically equivalent orbitals have) to have different populations so the system distorts itself. The distortion will always be along a single axis and per defintion this axis is always the $z$ axis. Thereby, ligands on the $z$-axis (negative charges in the most simple crystal field model) move away from the metal centre stabilising any orbital with a contribution in $z$ direction and most significantly the $\mathrm{d}_{z^2}$ one.

The effect is greatest in this case because the $\mathrm{e_g}$ orbitals actually point towards the ligands thus rendering them most susceptible to distance effects.

All other cases

In all other cases, the effect is much weaker. This is mainly because no other orbitals are known to point directly at the ligands; even for tetrahedral complexes the $\mathrm{t_2}$ orbitals point in-between them. Therefore, moving a ligand away slightly has much less effect on the electronics of the system.

What exactly happens really depends on the system itself and how it can be stabilised. If you are speaking of an octahedral complex, it will usually distort itself to $D_{\mathrm{4h}}$, i.e. equivalently to the former case because that is the distortion that keeps the highest possible symmetry. In that case again, one would say that the ligands in $z$ direction are moved away. Otherwise you need to figure it out by drawing, thinking and guessing or by studying crystal structures.



Tl;dr

Instead of memorising a list remember the first case and $z$-distortion. Ignore all the other cases for being weak.