Obtain transfer function from signal flow graph
I am afraid that for complex systems finding such free standalone tool would be hard. Obviously since you mentioned MATLAB, you should use Simulink then or this tool:

Fully Automated Flowgraph Analysis Tool. Manual to be found here. 

Alternatively you might look into this Mason.m tool. Or this masonry tool.

I know that these guys also did similar work once.