What is the class of this optimization problem?
After some research on this problem I've realised the model I've developed was incorrect. This is because I introduced the weight vectors in a wrong manner. 

Essentially, in my first model(the one this question was based on), the weight vectors were applied on the target vectors and on the input vectors that went into the model. Suppose this is right...because the model converges to the target => on the long term this will behave like a linear transformation: $F(w\mathbf{x})=wF(\mathbf{x})$
Introducing this into $E_u$ we get that $w$ doesn't even matter. Diving more deeper I realized this was a property of space. I was computing an $L_2$ norm...by defining an error as the distance of the model vector from the target. No matter how you strech you transform all the points (target and model) in the same manner &lt;=> the relative positions between the points don't change (the errors might be bigger or smaller, but the order relationship between the errors will be preserved).

The solution was to update my model to use instead the weighed euclidean distance. This means: 
$$E_u = \dfrac{1}{N_u}\sum_{i=0}^{N_u-1}\left\lVert x(t_{i+1}) - F_D(\{x(t_j)_{j=0,i}\})\right\rVert_D$$

I've written $F_D$, because at some point, my model does a query using a Ball Tree, which depends on D. I don't know theoretically why this works, but I can say that experimentally, it works (in the sense that the model outputs things which I expect and are reasonable).