Calculating Mixing Enthalpy in GROMACS
GROMACS won't be able to calculate the enthalpy of mixing directly but using the energy function in GROMACS you will be able to get energy, pressure, and volume of the simulations. 

Going through the paper that you posted it looks like the ran multiple simulation boxes for the different components and mixtures. They did this because you will not be able to get U,P, and V for individual components in GROMACS. So in order to calculate the enthalpy of mixing for a 2 component mixture they ran 3 simulations: one of the mixture, and one of each component by themselves. 

Once you have the simulation boxes for the different components and mixtures it should be pretty straight forward to calculate the enthalpy of mixing.

Just a side note: I am always a little nervous calculating thermodynamic expressions using pressure from GROMACS because the pressure will fluctuate so wildly. So if you're going to make sure you're careful about your equilibration run and your NPT equilibration run as well. 

Below are the GROMACS manual for energy which you would use to get U,P, and V from your simulation. The next is a tutorial for an NPT run from Virginia Tech and their tutorials are really well written and should help as a starting point and finally the last is a chat between a user and GROMACS developer discussing pressure fluctuations in NPT. 

http://manual.gromacs.org/documentation/current/onlinehelp/gmx-energy.html

http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin/gmx-tutorials/lysozyme/07_equil2.html

https://mailman-1.sys.kth.se/pipermail/gromacs.org_gmx-users/2011-May/061370.html