how does xgboost handle inf or -inf values?
Since (most) tree-based methods only care about the ordering of values in each feature, replacing your infinite values with very large values (larger than any finite value of the feature) is fine.  Of course, you'll have to do some thinking of whether df[col1]/df[col2] should actually be treated as $\pm$inf when col2 is zero, and nans can just be left in for xgboost.