Signal Flow Graph Problem
Mason's rule is a nice tool that converts the system into a single transfer function. Once you get the transfer function, you can covert it to the state space representation if and only if the transfer function is proper. Mason's rule is done as follows:

1- Loop gain

\$ L_1= -\frac{1}{s}, L_2 = -\frac{1}{s}\$.

2-Forward-path gain

\$ P_1 = (1)(-1)(1)(-1)(1) = 1.\$

3-Nontouching loops

\$ L_1L_2\$

Mason's rule is 

\$ G(s) = \frac{Y(s)}{U(s)} = \frac{\sum_k P_k\Delta_k}{\Delta}\$

where 

\$ \Delta  = 1- (L_1 + L_2) + (L_1L_2) = 1 + \frac{2}{s} + \frac{1}{s^2}\$ and \$ \Delta_1 = 1.\$

The single transfer function is then  \$ G(s) = \frac{s^2}{s^2 + 2s^2+1}\$. It is clearly that the transfer function is proper that is the polynomial's degree of the numerator is less than or equal the denominator's polynomial's degree. Rewrite the system, we get 

$$
(s^2+2s+1)Y(s) = s^2 U(s)
$$

In the time domain, we get 

$$
\ddot{y}(t) + 2\dot{y}(t) + y(t) = \ddot{u}(t)
$$

The transfer function has this form 
$$
G(s) = \frac{Y(s)}{U(s)} = \frac{ b_0 s^2 + b_1 s + b_0 }{ s^2 + a_1 s + a_2 }
$$

Controllable canonical form can be utilized to represents the system as follows:

$$
\begin{align}
\dot{x} &amp;= 
\begin{bmatrix}
0 &amp; 1\\
-a_2 &amp; -a_1
\end{bmatrix} +
\begin{bmatrix}
0 \\ 1
\end{bmatrix}
u \\
y&amp;=
\begin{bmatrix}
b_2-a_2b_0 &amp; b_1-a_1b_0
\end{bmatrix}
+ b_0u
\end{align}
$$

where \$b_0=1,b_1=0,b_2=0,a_1=2,a_2=1\$. The state space representation of the system is 

$$
\begin{align}
\dot{x} &amp;= 
\begin{bmatrix}
0 &amp; 1\\
-1 &amp; -2
\end{bmatrix} +
\begin{bmatrix}
0 \\ 1
\end{bmatrix}
u \\
y&amp;=
\begin{bmatrix}
-1 &amp; -2
\end{bmatrix}
+ u
\end{align}
$$

Using Matlab, we can test the transfer function, therefore, the code is 



syms s
num=[1 0 0];
den=[1 2 1];
G=tf(num,den);
step(G)


Now let us test the state space representation that we have formed. 



b0=1; b1=0;b2=0;
a1=2;a2=1;

A=[0 1; -a2 -a1];
B=[0;1];
C=[b2-a2*b0 b1-a1*b0];
D=b0;
sys=ss(A,B,C,D);
step(sys)


which is the same result we've got.