What is a possible software for simulating binary star systems?
MESA

It is designed specifically for stellar evolution. There are many modules that you can play around with, including binary interaction.