Terminology to distinguish between ML methods and optimization methods (PSO, ACO..)
In a wider context every machine learning method can be re-cast as some type of optimisation problem. For example for Neural Networks the associated optimisation problem is &quot;find the weights which minimise some loss function of the data given an architecture&quot;. This is solved using back-propagation (which is a layered gradient-descent method) and when minima of the loss function are found we say the systen has &quot;learned&quot;.
On the other hand there is nothing stopping us from re-casting optimisation problems as &quot;learning&quot; problems
.
A possible differentiation (based on what you mention in the question) is between analytic methods and non-analytic methods.
A. Analytic Methods: Gradient-based, Primal-Dual etc... (eg NNs, SVMs)
B. Non-Analytic Methods: Particle systems, Genetic/Evolutionary systems, Simulation methods, Stochastic methods, etc..
The basic differrence is both whether the form of the objective function is known and if analytic methods (eg gradient-descent vs particle systems) are used to find optima.
Again, there is nothing, in principle, stopping us from using these non-analytic optimisation methods to do machine learning.