What is the difference between doing a PhD in a small university versus a big reputed university in Germany?
Does the size of a university play any role in someone's PhD research experience in Germany?


First of all, I am assuming that your PhD does not contain any course components, because that’s the norm in Germany. Otherwise, it gets complicated and strongly depends on the PhD programme in question.

When doing a PhD, the vast majority happens within your group. The money and facilities you have available here mostly depend on what funding, facilities, etc. the professor managed to acquire. This might slightly correlate with the power of the university, but the fluctuations between groups are much higher. It makes for more sense to judge the group in this respect.

Aspects of the location that affect your PhD from outside your group will be:


Are there more groups in your general field? If yes, there will be more opportunities to interact with visiting researchers, get quick feedback on certain aspects, have a secondary supervisor who at least somewhat knows what you are doing, etc. It is a small drawback if your group is isolated at your university. (Note that you sometimes have to take neighbouring universities into account here. There are not few cases, where parts of the next university are easier to reach than other parts of the same university.)
Does the university have all the facilities that go beyond what a single group usually has, like a workshop to create special devices, analysing facilities, a supercomputer, etc.
Will you have any teaching duties and if yes, what are they? You probably want to gather some relevant teaching experience, but not spend too much time on this.
Are there enough students for your group to regularly have bachelor and master students doing there thesis? This may give you the opportunity to gather advising experience and benefit from the research output of delegated side projects. Depending on your field, however, this may also be just a huge time sink.
Many if not all universities offer workshops or similar for PhD students about teaching, scientific writing, searching for postdoc jobs, writing grants, etc.


You will get more reliable information on most of the above when you directly ask the professor you are applying to. Moreover, most of this depends on your department or faculty, and not on the university. For example, if you are doing your PhD in computer science, it does not matter whether your university hosts faculties or departments of social sciences, engineering, law, medicine, agriculture, biology, etc. (unless, of course, you are doing research about direct application to those fields). In particular some faculties have run pretty much independently over centuries; they just share a common administrative backbone, location, and name and sometimes some of their researchers collaborate with each other.

At least in Germany, the variability between individual groups within a department is higher that the variability between individual departments and faculties within a university, which in turn is higher than the variability between universities.
The main thing you probably need to consider will be whether your future potential employers are aware of this, but that really depends on what you want to do with your degree and where:
Some people just like easy categorisations like having high- and low-quality universities, even though these are completely oversimplifying things.