What material can I use as a dielectric in my own high voltage capacitors?
You forgot to mention how much capacitance you need. You see, this aluminium ruler on my desk and the steel whiteboard 2 m away make an excellently safe 50 kV capacitor. It just has a very low capacitance. You might want to make up your mind on that :)



I don't see how the pump approach would even get you close to the cheapness of two 50 kV capacitors (the situation is not exactly as bad, but 30€ is still money). Also, this sounds like you're neglecting the hard problem here (deionizing water, and making sure all the ducting doesn't introduce ions), and focus on the easy stuff (pumping water).


  Waxed paper is, as far as I know, basically the same as baking paper. 


If that was the case, your cookies would taste like candle and be stuck to the paper.

It's not; waxed paper is mainly a layer of paraffin (or some other form of synthetic wax with low water content, but paraffin is really the prime choice) applied to a paper "core" for stability.

You can probably buy that or make that yourself, but you'd honestly would want to work very cleanly, very dry, and very uniformly so that you don't get any point in your piece of wax paper that has potential for a breakdown, as lightning shooting through a mixture of wax, a paper wick and high voltage is probably a very good method to start a fire that produces a lot of thick smoke very quickly.

You could at least address that by grossly overdimensioning your wax thickness, but that would reduce the capacitance of your capacitor (that's basically inversely proportional to the dielectric thickness).

Also, remember, including all the effort you put into manufacturing things precisely and all the tools and supply you need for this one-off effort, you're still trying to go below ~100€. I think you'll find this a practical problem once you really sit down to do a rough cost calculation.


  Do you have an idea on how to achieve my goal?


Yes:


  Wikipedia lists a lot of chemical substances with crycptic names that have a high dielectric strength.


Well, investigate these!

Honestly, waxed paper is probably a good starting point: if you can find a perfectly air-gap and -bubble-free way of constructing a wax-flooded (ie. paraffin-flooded) capacitor, you might get around acquiring or making your own wax paper.