How to approach All vs All classification problem
I think you can carry out a usual multiclass classification instead of manually carrying out the one-VS-all strategy (i.e. in the for loop), provided that you can generate the multilabels target(i.e. all the possible combinations of diseases you said). So, I would do something like:

generate the correct labels --&gt; if you have n possible deseases combinations, you should have in an end n target labels (so using the apply-lambda expression generates n binary columns for you, which you can also convert to a single numeric column with n possible integer values).
Below you can find a way to get your final column of target values (an integer value for each possible combination); using the wines dataset as an example where I also consider all possible combinations (here a flavour type is similar to a possible disease in your case). I guess you have a dataframe like this:


so you can create a dict mapping the possible combinations to an integer target value (for the multiclass classification):
target_labels_dict = {tuple([1, 0, 0]): 0, tuple([0, 1, 0]): 1, tuple([0, 0, 1]): 2, tuple([1, 1, 0]): 3, tuple([1, 0, 1]): 4, tuple([0, 1, 1]): 5, tuple([1, 1, 1]): 6}

and you have this:

where for instance the tuple (1, 1, 0) means 'acid' and 'sweet'...
and finally, create your target column:
target_flavours_array = np.zeros(len(y_train)).astype('int') 

for ind in y_train_df.index:
   combination_values = tuple(y_train_df.iloc[ind])
   target_flavours_array[ind] = target_labels_dict[combination_values]

getting something like:


use the softmax (or its analogous sofprob function check it out here) with the xgboost instance --&gt; below a quick example of this (training on the original 3 classes dataset for the shake of the example):

from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
import numpy as np 
import xgboost as xgb

dataset = load_wine()
X_train, X_test, y_train, y_test = train_test_split(dataset.data, 
dataset.target, test_size=0.1, random_state=42)

dtrain = xgb.DMatrix(X_train, label=y_train)
dtest = xgb.DMatrix(X_test, label=y_test)    

params = {
    'max_depth': 4,  # maximum depth of each tree
    'eta': 0.2,  # learning rate
    'objective': 'multi:softprob',  # for multiclass problem, providing probabilities
    'num_class': len(np.unique(dataset.target))} # number of classes

iters_number = 30
xgb_model = xgb.train(params, dtrain, iters_number)
xgb_model.predict(dtest)

and, the result is the prediction probability for each possible class (diseases-combination in your use case):

here you can select the highest probability value for each row.