That said, if you are allergic to lactose or have a lactose intolerance, it will not prevent you from losing body fat. Weight gain is an energy imbalance issue, meaning that you are consuming more food and fluid than you are using.