Safe voltage for hydrolysis of water - teacher demo
The reason I asked about the student ages was that I wondered just how much chemistry teaching you could pile into this. But given "middle school" I gather this is more about just trying to get their attention at all and to try and get them self-motivated. The social milieu is so distracting during that period of time that this alone can be a frustrating process.

Here, all I want to do is to provide something to inform you about the required voltage and to discuss some thoughts I have about getting younger students interested in pursuing science with their own energy and motivation (and not just pushed into it.)

The first step in working out the required voltage is something called the Gibbs energy. I won't belabor it much here, but refer you instead to this answer for determining the standard Gibbs free energy for water. There, you can see that at \$25\,^\circ\text{C}\$ water's value is \$-237.13\:\text{k}\frac{\text{J}}{\text{mol}}\$. Avogadro's number, (\$6.02214086\times 10^{23}\:\frac{\text{molecules}}{\text{mol}}\$), provides the number of molecules per mole. Knowing that, and knowing you need two electrons per molecule (one for each hydrogen atom), this works out to:

$$\begin{align*}V&amp;=\frac{-237.13\:\text{k}\frac{\text{J}}{\text{mol}}}{2\:\frac{\text{electrons}}{\text{molecule}}\cdot 6.02214086\times 10^{23}\:\frac{\text{molecules}}{\text{mol}}\cdot 1.60217662\times 10^{-19} \frac{\text{coulombs}}{\text{electron}}}\\\\&amp;\approx 1.22883962\:\text{V}\end{align*}$$

This does NOT take into account other factors due to inefficiency or the fact that you are starting with a liquid (lower entropy) and going to a gas (higher entropy.) But it does tell you in broad strokes that you don't need a high voltage. \$2-3\:\text{V}\$ is more than enough and certainly \$6\:\text{V}\$ would be an over-voltage beyond any factors related to inefficiencies, enthalpy, or entropy.

It's not clear to me why you aren't using a Hoffman apparatus for the demo using larger current compliances than a \$9\:\text{V}\$ battery can provide. It's the right piece of equipment to use, if you can afford it.

By the way, in 6th grade we were allowed to make smoke bombs (I used iodine crystals as part of my choice there.) I think students at that age or a little older need fire, smoke, light. Long term "future value" ideas don't motivate because it is too far out into the future and students have to "bang their heads against a wall" to learn and it's not fun unless they can get some immediate personal benefits. (You can't get them started with "when you get older you will be glad you learned this stuff.") I believe your job should NOT be to eliminate all risk in this process, but instead to mitigate the risk as much as reasonably possible without losing the educational and motivational values. In my last year of high school chemistry, I made nitroglycerin and mercury fulminate (in class in a school with about 2,000 students in all four grades.) We also made large crystals out of aluminum foil wrapped around styrofoam balls and put together with wood rods. We then irradiated the crystals using a klystron on the table in the front of the room while we ran around with watt-meters measuring diffraction. These things made a huge difference in my life.

Of course, being careful applies to you as much as it should for the students. So do be careful when you are generating a lot of gas. Providing enough voltage will be the least of your problems. Current compliance is important and getting a consistent conductance in the water (using salts) and avoiding the generation of gases you do not want (such as chlorine, brown's gas, etc.) Adding sodium chloride (salt) to the water to improve the conductivity will yield chlorine mixed with the hydrogen, for example. My recommendation is that you research this entire topic and learn from the pros. Making gas fast carries risks and you need to be aware of them, how to detect problems, and how to properly use the results with lowered (but always non-zero) risks to yourself and others. Put in the time, both in studying the mistakes and successes of others but also in performing your experiments privately to make sure you know the range of results to expect before performing them in public and in class.