Applying Expectation Propagation to Factor Graph
The way you are doing is not quite correct. Take a look at these tutorials 
 -  http://homepages.inf.ed.ac.uk/csutton/publications/ep-tutorial.pdf 
 -  http://web.engr.illinois.edu/~khashab2/learn/ep.pdf 

Go over the algorithms which show the EP procedures. Also note the way they explain the projection of the distributions using KL projection.