How to do a color separation with more than 3 primary colors How would one do a color separation if there are more than 3 color primaries, or the primaries are nonstandard. In Standard  CMYK conversion K is relatively easy conceptually to figure out. Its just a constant value off all the channels. 

How would one construct a CMY + orange for instance. Or in the case of a led array with say Red, Yellow, Blue and Violet for example. How would one approach this problem. 

I'm fine with a conceptual answer no need to do LAB to end result conversion for example. Though that would be nice.
The mathematical approach to this is to represent the other colors/light sources in terms of your standard primaries in the color space. Represent your target color and available light sources as vectors with, say, the RGB color space as the basis. Figuring out how to display your target color using your available sources then becomes a change of basis, a standard linear algebra operation, where the target basis consists of your LEDs/nonstandard primaries.

If the dimensions of the source and destination bases are different, the change of basis matrix won't be square and thus won't be invertible. In that case, you need to solve a system of linear equations, expressed in matrix form:

$$c_{RGB} = [r, y, b, v] \cdot c_{RYBV}$$

Where $c_{RGB}$ and $c_{RYBV}$ are the coordinate vectors of your desired color expressed in terms of RGB and custom primaries, respectively, and $r$, $y$, $b$ and $v$ are your custom primaries (basis vectors) in RGB coordinates.

Solving this system is another common linear algebra operation, and there are a variety of numerical and symbolical algorithms to do so. The system is over-constrained, so solutions will likely have a free variable, however since we can't physically produce negative light, nor do the primaries have infinite dynamic range, for most practical purposes each coefficient will have to be limited to be within the $[0, 1]$ range. This will render many colors unrepresentable under certain sets of primaries.