What does "to be enjoyed with all rights and privileges pertaining thereto" mean on a French diploma?
That phrase is clearly not France-specific, as @ff524 mentioned. In order to add to and further illustrate the nice answer by @vonbrand, I will share the following paper, which discusses Roman origins and Medieval expressions of the relevant phase(s): http://www.jstor.org/stable/1844716.

In addition to some comments and answers for the above-mentioned most likely duplicate question, I would add that modern practical meaning of this phrase significantly depends on graduate's field of study and institution they graduated from.

In regard to the field of study, rights and privileges might include (beyond the implied rights and privileges to say that one graduated with specific degree from a particular institution, to wear the institution's regalia, to be referred to as a Dr. [for Ph.D. graduates], etc.): to be able to practice in specific regulated fields, such as medicine and law (upon satisfying additional conditions, such as attending medical residency or passing specific state's bar examination, correspondingly).

In regard to the institution, some rights and privileges include to participate in alumni activities, to retain institution's e-mail address, to get discounts on various products and services as well as on attending individual classes and, even, enrolling into certain degree programs at one's alma mater.