Incorrectly applying random forest model?
I think you correctly identified the issue: if your model tries to classify whether an employee stays or leaves, by definition every employee "stays" as long as they are an employee in the company. 

A possible direction would be to design the response variable as "does the employee leave within a year?" (or any specific period of time). This way you can use the records of past employees who have left and past employees who have stayed across time, i.e. you can have several instances corresponding to the same employee at different times. You could add features such as "had a raise or promotion in the past 2 year" for instance.