What is the difference in xgboost binary:logistic and reg:logistic
There is no difference in the algorithm, they are the same. When you use  reg:logistic the default evaluation metric is rmse, but you can change this parameter.

The error rate and the rmse may differ depending on the distribution of your output, as the error rate uses a limit of $0.5$ if you have the output values concentrated in $0$ or $1$ it will be much smaller than rmse, even though its correlated metric the model can be very different, the application will depend on your problem.

To evaluate performance of logistic regression models is usual to calculate auc, logloss, accuracy, pseudo-$R^2$ and some others.