State Space model for a MIMO, coupled system
You misunderstood state-space representation. While the standard form you've posted indeed targets linear systems but this doesn't mean the representation can't handle nonlinear system. Probably one of powerful tools of this representation is its capability for handling nonlinear system. Let's see how the system is represented in the state space (i.e. no matrix form). 

Let's define \$q_1=x, q_2=\dot{x},q_3=y, q_4=\dot{y},q_5=\theta, q_6=\dot{\theta}\$, therefore, we obtain

$$
\begin{align}
\dot{q}_1 &amp;= q_2\\
\dot{q}_2 &amp;=  F_E\cos(\psi)\sin(q_5) + F_E\sin(\psi)\cos(q_5) +F_s\cos(q_5)\\
\dot{q}_3 &amp;= q_4\\
\dot{q}_4 &amp;=  F_E\cos(\psi)\cos(q_5) - F_E\sin(\psi)\sin(q_5) - F_s\sin(q_5) \\
\dot{q}_5 &amp;= q_6\\
\dot{q}_6 &amp;=  F_E\sin(\psi)(l_1+l_n) - l_2F_s\\
\end{align}
$$
Now the system can be represented generally as 

$$
\dot{q} = f(q, u), \quad q \in \mathbb{R}^3,  u \in \mathbb{R}^3
$$
which can be solved numerically. I can solve the system if you provide me with the actual parameters of the system.