Electrochemistry conditions for voltage to increase in a galvanic cell
Yes, your voltage will change in accordance with the Nernst equation

$$\Delta E_\text{cell} = - \frac{RT}{zF}\ln Q_{\text{rxn}}$$

So, in your copper-zinc battery, where

$$\ce{Cu^2+(aq) + Zn(s) -&gt; Cu(s) + Zn^2+(aq)}$$

with a standard $E^\circ_\text{cell} \approx \pu{1.10 V},$ setting the $\left[\ce{Cu^2+}\right] = \pu{4 M}$ should raise your cell potential by

$$\Delta E_\text{cell} = -\frac{RT}{2F}\ln\frac{1}{4} = \pu{+0.0178 V}$$

for a new cell potential (at $\pu{25 °C})$ of

$$E_{\text{cell}} = \pu{1.10 V} + \pu{0.0178 V} \approx \pu{1.12 V}$$