How to create RGB-colors with equal vector norm?
So how would one generate RGB vectors with a constant norm that are still valid RGB values? ($0 \le r,g,b \le 255$)


There is a simpler algorithm. Let $L$ be the given norm.


Verify that $0\le L\le 255\sqrt3$.
Let r be a random number between $\sqrt{L^2-\min(2\cdot255^2, L^2)}$ and $\min(255, L)$.
Let $M=\sqrt{L^2-r^2}$. Let g be a random number between $\sqrt{M^2-\min(255^2, M^2)}$ and $\min(255,M)$.
Let $b =\sqrt{L^2-r^2-g^2}$.




Some minor adjustment is needed if $r,g,b$ should be integers, in which case only approximate solutions can be found in general.



Here is the right computation if we want to find the range of $\theta$ and $\phi$.


If $0\le L\le 255$, then $0\le\theta,\phi\le\pi/2$
If $255\lt L\le 255\sqrt3$, let $M=\sqrt{\max(0, L^2-2\cdot255^2})$.
we have
$$\begin{align}
&amp;\quad\quad\begin{cases}b = L \sin(\theta) \cos(\phi)\\
r = L \sin(\theta) \sin(\phi)\\   
g = L \cos(\theta)\\
0\le b,r,g\le 255\\
0\le\theta,\phi\le\pi/2
\end{cases}\\
&amp;\Longleftrightarrow
\begin{cases}b = L \sin(\theta) \cos(\phi)\\
r = L \sin(\theta) \sin(\phi)\\   
g = L \cos(\theta)\\
\sqrt{\max\left(0,(L\sin(\theta))^2-255^2\right)}\le b,r\le \min(255,L\sin(\theta)) \\
M\le g\le 255\\
0\le\theta,\phi\le\pi/2,\, \theta\not=0
\end{cases}\\
&amp;\Longleftrightarrow
\begin{cases}
\sqrt{\max\left(0,1-\left(\dfrac{255}{L\sin(\theta}\right)^2\right)}\le \cos(\phi)\le \min\left(1, \dfrac{255}{L\sin(\theta)}\right)\\
\sqrt{\max\left(0,1-\left(\dfrac{255}{L\sin(\theta}\right)^2\right)}\le \sin(\phi)\le \min\left(1, \dfrac{255}{L\sin(\theta)}\right)\\
\dfrac{M}{L}\le \cos(\theta)\le \dfrac{255}L\\
0\le\theta,\phi\le\pi/2,\, \theta\not=0
\end{cases}\\
&amp;(\text{the inequalities on}\cos\phi\text{ put on the same restrictions on }\phi\\
&amp;\text { as the inequalities on }\sin\phi)\\
&amp;\Longleftrightarrow
\begin{cases}
\arccos\left(\min\left(1, \dfrac{255}{L\sin(\theta)}\right)\right)\le \phi\\
\phi\le\arccos\left(\sqrt{\max\left(0,1-\left(\dfrac{255}{L\sin(\theta)}\right)^2\right)}\right)\\
\arccos(\dfrac{255}L)\le\theta\le\arccos(\dfrac{M}{L})
\end{cases}
\end{align}
$$


The formula above tells a way to generate RGB vectors with a given norm $L$. Select $\theta$ and $\phi$ in the range given by the inequalities. Compute $r,g,b$ accordingly.

In particular, if you take $L=255\sqrt3=441.67$, then $\dfrac{255}L=\dfrac{M}L=\dfrac3{\sqrt3}$. $\theta=\arccos(255/L)=0.9553$. Since $\dfrac{255}{L\sin(\theta)}=0.7071$, we will get $\arccos(0.7071)\le \phi\le\arccos(0.7171)$. So $\phi=\pi/4$ and we will arrive at $b=255$, $g=255$ and $r=255$, which is the only possible combination of RGB vector for that value of $L$, as expected. 



There have been several computational mistakes in the question. There has been a few in the previous versions of this answer as well. The comments below this answer might have become irrelevant.