Constrained optimization topics
This can be formulated as a mixed-integer quadratically constrained quadratic programming (MIQCQP) problem, integer programming with quadratic constraints and a quadratic objective function.   You can find off-the-shelf  solvers out there, and I think some of them support MIQCQP.  Unfortunately MIQCQP is a pretty hard problem, so it's hard to know whether this will be effective, for the size of problem you have.  But it could be worth a try.

You can encode the max by the two constraints $p_j \ge 0$, $p_j \ge (\sum_i X_ij - D_j)^2$, where $p_j$ is a variable representing $p(X,j)$.  Since you're going to minimize a sum of $p_j$'s, this suffices (the minimization objective will chose the smallest $p_j$ that satisfies both inequalities, which is the max).



Alternatively, if you're willing to remove the "squaring" in the definition of $p(X,j)$, this could be formulate as an instance of integer linear programming.