Determining change in entropy and enthalpy during mixing of gases
Your equation for $\Delta S_{system}$ is incorrect.  The correct equation is $\Delta S_{system}=\int{\frac{dq_{rev}}{T}}$.  I hope you understand the difference, and the fact that the subscript "rev" refers exclusively to a reversible path between the initial and final states of the system.  To get that, you need to forget entirely about the actual irreversible path, and instead focus on the two end states.  You then have to devise a reversible path between these same two states so that you can determine the value of the integral.  For your problem, this recipe will yield a positive value for $\Delta S_{system}.$

Regarding the enthalpy change, you are aware that the enthalpy of mixing of ideal gases is zero, correct?