total transfer function of a LTI System with feedback
My answer comes a little late, but I think it should still be interesting.
I took the block diagram from you (@Newbie010) as a basis and added some labels which are helpful for the calculation. My method gets along here without any simplification.
At this point I ask you to check if what I have done is correct. I did not find any mistake in my calculation, but a critical consideration would be appropriate.

Block diagram with new labels:

Based on this picture, the following equations can now be determined:
$$X(s)+C(s)=B(s)=A(s) \tag{1}$$
$$D(s) = B(s) - H_1A(s) \tag{2}$$
$$E(s) = H_4Y(s)+H_1A(s) \tag{3} $$
$$F(s) = E(s) + H_3D(s) \tag{4}$$
$$Y(s) = H_5F(s) \tag{5} $$
Other important and useful equations:
$$ F(s) = \frac{Y(s)}{H_5} \tag{6}$$
$$ C(s) = H_2F(s) \tag{7}$$
$$ C(s) = H_2\frac{Y(s)}{H_5} \tag{8}$$
The aim now is to have only inputs and outputs in its transfer function $H(s)$. Important here is this connection $H(s) = \frac{Y(s)}{X(s)}$
I now begin to generate $H(s)$ by starting with equation 5 and using the relations to other equations in turn. This is straight forward:
$$ \begin{align} Y(s) &amp;= H_5F(s) \\ 
&amp;= H_5\left(E(s)+H_3D(s)\right) \\
&amp;= H_5\left(H_4Y(s)+H_1A(s)+H_3D(s)\right) \\
&amp;= H_5\left(H_4Y(s)+H_1A(s)+H_3B(s) - H_3H_1A(s)\right) \\
&amp;= H_5\left(H_4Y(s)+H_1\left(X(s)+C(s)\right)+H_3\left(X(s)+C(s)\right) - H_1H_3\left(X(s)+C(s)\right)\right) \\
&amp;= H_5\left(H_4Y(s)+H_1\left(X(s)+H_2\frac{Y(s)}{H_5}\right)+H_3\left(X(s)+H_2\frac{Y(s)}{H_5}\right) - H_1H_3\left(X(s)+H_2\frac{Y(s)}{H_5}\right)\right) \\
&amp;= H_5H_4Y(s)+H_5H_1X(s)+H_5H_1H_2\frac{Y(s)}{H_5}+H_5H_3X(s)+H_5H_3H_2\frac{Y(s)}{H_5}-H_5H_1H_3X(s)-H_5H_1H_3H_2\frac{Y(s)}{H_5} \\
&amp;=Y(s)\left(H_5H_4+H_1H_2+H_3H_2-H_1H_3H_2\right)+X(s)\left(H_5H_1+H_5H_3-H_5H_1H_3\right)
\end{align} $$
Therefor:
$$ Y(s)(1-H_5H_4-H_1H_2-H_3H_2+H_1H_3H_2) = X(s)(H_5H_1+H_5H_3-H_5H_1H_3) $$
It follows:
$$ H(s) = \frac{Y(s)}{X(s)} = \frac{H_5H_1+H_5H_3-H_5H_1H_3}{1-H_5H_4-H_1H_2-H_3H_2+H_1H_3H_2} $$
Now I wanted to test if what I have done can be confirmed by a simulation. Therefore I converted the block diagram in Matlab. Also I realized my generated transfer function as a block in Matlab, the results were the same!
For the simulation I used the following blocks:
$$ H_1 = 5, H_2 = \frac{1}{s}, H_3 = \frac{1}{s+2} $$
$$H_4 = 7, H_5 = \frac{1}{s}$$
If I now insert these values into my TF and let the overall expression be simplified once with the use of Wolframalpha I get the following for $H(s)$:
$$\frac{5s+6}{s^2-10s-20}, s\neq 0, s \neq -2$$
In Matlab I implemented your block diagram with the corresponding values and my simplified transfer function. Judging by the simulation, the results are the same.
I hope that my answer meets the requirements and hope that I could help a little bit with this. Thanks a lot!