Common Techniques to Generate from a Regression Neural Network Model
Let $E_\phi : D\rightarrow \mathbb{R}$ be your trained differentiable regression model, where $D$ is the data space, e.g. images. Let $G : \mathbb{R}^d\rightarrow D$ be some generative model or decoder from a latent space (e.g., GAN or decoder from a VAE).

Suppose we want to find $x\in D$ such that $x = \min_y E_\phi(y)$. Then there are two obvious ways:


Gradient descent in the data space: i.e., solve $x = \min_y E_\phi(y)$ via iterating $ x_t = x_{t-1} - \eta\nabla E_\phi(x_{t-1}) $.
Gradient descent in the latent space: i.e., solve $z = \min_u E_\phi(G(u))$ via iterating $ z_t = z_{t-1} - \eta\nabla (E_\phi(G(z_{t-1}))) $.


Both are very common.
The main benefit of (2) is that (a) you are more likely to get a "reasonable" $x=G(z)$ because the generator $G$ was trained to give you one (i.e., $x$ should look like it was from $D$, which is not guaranteed in (1)), (b) $d$ is often lower dimensional than $|D|$, making the optimization easier, and (c) it may be impossible to even do (1) (for instance, in molecule generation, we can do (2) but not (1) due to the discreteness problem). Nevertheless, there is a large drawback: the need to learn $G$, which is itself non-trivial to obtain. (Hence why for e.g. generating adversarial examples, (1) is more popular.) 

One might also want to avoid gradient-based optimization in some cases, in which case I often see Bayesian optimization approaches being used.