Relation between phase and magnitude
In general, you'd have to project the light from the sun (Assuming we're talking about objects in our solar system) onto the 3D surface and project it onto a 2D surface. This can be accomplished via a ray tracing algorithm. 

This procedure will give you the fraction of the surface illuminated.

Now if you are concerned about the fraction of the illuminated surface that is reflected in a specific directly -- so that it is "viewable" from that direction -- you have to do the ray tracing in 3D from the source (Sun) to the object and on to the viewer. There packages you can google that will do this type of ray tracing if you can model the objects in 3D...