How can I properly calculate the isoelectric point (pI) of amino acids?
Isoelectric point of an amino acid is the $\mathrm{pH}$ at which the molecule carries no net charge[1]. It can be calculated by the average of the relevant $\mathrm pK_\mathrm a$ values as you have mentioned.
Your confusion seems to stem from choosing the relevant $\mathrm pK_\mathrm a$ values. For this we should refer to the titration curve of the amino acid.
For a neutral amino acid[2]:

From the curve we can infer that the $\mathrm{pI}$ is simply the average of the two $\mathrm pK_\mathrm a$ values of the carboxylic acid and the amino group.
For a basic amino acid[2]:

From the curve we can infer that the $\mathrm{pI}$ is simply the average of the two $\mathrm pK_\mathrm a$ values of the two amino groups. The $\mathrm pK_\mathrm a$ of the carboxylic acid group is not relevant.
For an acidic amino acid[3]:

From the curve we can infer that the $\mathrm{pI}$ is simply the average of the two $\mathrm pK_\mathrm a$ values of the two carboxylic acid groups. The $\mathrm pK_\mathrm a$ of the amino group is not relevant.
Here are examples for all three cases:

References:

Wikipedia

Titration curves for neutral and basic amino acids

Titration curve for acidic amino acids