Should I include a "prestigious" invited-only conference/workshop on my CV?
I'm afraid of appearing over-proud of a minor deal.


In general, you should not worry about this.  A CV is a document where you are supposed to "sell yourself".  Everyone expects you to highlight those things that show your ability, experience, and recognition by prominent people in the field.  

This makes many people uncomfortable, since talking up our own achievements is not the norm for most of us.  However, it is the right thing to do for a CV.  List anything that provides evidence that you are a strong candidate.

Is it possible to have a CV that appears "over-proud"?  Yes, if you did something like clearly exaggerate your experience, or use boastful language.  But merely factually listing something you achieved or participated in is not going to create that impression.

As for this specific workshop, definitely put it on your CV.  The fact that prominent researchers thought you were worth including in this event speaks well of you.  I wouldn't call that a "minor deal", especially for a PhD student.