PhD thesis - Names of appendices
In a mathematics paper or thesis, the most common practice is simply to include all proofs in the body of the document, each one immediately following the statement of its corresponding theorem or lemma.  So you should seriously consider just doing that.

Placing proofs in an appendix might be appropriate if they are really ancillary to the main thrust of your thesis.  For instance, maybe there is a result that is not really relevant to the rest of the thesis, but you've included it simply because it is interesting.  Or you state a result that is well known, but you have written your own proof as an exercise.  Or you have multiple proofs of the same statement; you might want to place one of them in the body, and move the other(s) to the appendices.

But in any case, I would think there would not be more than a few proofs that would fit these criteria.  As such, it would be most natural to place each proof in its own appendix, rather than to group them by body chapter or any other criterion.

You could give the appendices names like "Appendix A.  Proof of Theorem 1.2.3".  Or better yet, something more descriptive like "Appendix A.  Proof of the Snargleberg&ndash;Veeblefester Theorem (Theorem 1.2.3)".  Of course, you will want to use LaTeX's \ref to produce the theorem number.

It would be a convenience to the reader if in each appendix, you repeat the statement of the relevant theorem before giving the proof.

If you feel the number of lettered appendices A, B, C, D, etc are becoming excessive, you could move down the hierarchy.  Most thesis templates are based on the book document class, in which the top-level text element is the chapter.  So if you want, you could write

\backmatter
\chapter{Additional Proofs}
\section{Proof of the Snargleberg--Veeblefester Theorem (Theorem \ref{snargleberg})}

Blah blah blah.

\section{Proof of the Dumbledore Lemma (Lemma \ref{dumbledore})}

Expecto Patronum.


Producing something like:


  A. Additional proofs.
  
  A.1.  Proof of the Snargleberg&ndash;Veeblefester Theorem (Theorem 1.2.3)
  
  Blah blah blah.
  
  A.2.  Proof of the Dumbledore Lemma (Lemma 2.3.4)
  
  Expecto Patronum.


I disagree with Pete L. Clark's suggestion to just leave the appendices with non-descriptive titles like "Appendix A".  Using descriptive appendix titles will help keep the reader from getting lost.