Should we include Source and Load Impedance while calculating two port parameters?
you shouldn't, the whole point of a two-port transformation is that you can use the simplified model and connect sources/loads as you see fit and see how it behaves regardless of source and load if you add the load to the parameters you would have to uncouple it from the equations every time you get a new load.

So I would leave out the impedance of the source and load out of any h/y/z or whatever parameters you are getting.