How difficult is this matching-like problem?
Observing that there exists an optimal solution such that for any $i_1&lt;i_2$ , $f(i_1)\le f(i_2)$ (otherwise, we can swap $f(i_1)$ and $f(i_2)$), there is a dynamic programming algorithm solving your problem.
We sort $A$ and $B$ firstly. Suppose $A=\{a_1,\ldots,a_n\}$ and $B=\{b_1,\ldots,b_m\}$, where $a_1&lt;\cdots&lt;a_n$ and $b_1&lt;\cdots &lt;b_m$. Let $D(i,j)=\min_{f:\{a_1,\ldots,a_i\}\to\{b_1,\ldots,b_j\}}\sum_{j'\le j}(\text{number of nodes matched to }j')^2$, then $$D(i,j)=\min_k D(i-k,j-1)+k^2,$$
where the minimum is taken over all $k$ such that $a_{i-k}$ is able to be matched to $b_j$. The original objective is to compute $D(n,m)$. It can be done by this recurrence in $O(n^2m)$ time.