model predicting probability close to 50 for positive classes in imbalanced training dataset
More than the probability, what you can consider is the confusion matrix you get after applying the threshold on the probability. If, for example, your threshold is 0.5 and your model is perfectly classifying your validation examples, you shouldn't have a problem.

If you take average log loss as your training metric, with equal weights for positive and negative classes, your average probability values will get shifted towards the dominant side in the training set.

Summarizing, probability values in themselves may not be a cause of worry. Focus more on imbalanced data specific metrics like precision, recall(sensitivity) specificity, F1 score or Area Under the ROC curve.