Track underlying observation when using GridSearchCV and make_scorer
Firstly; this is a really clear, well written question. Kudos!

I think the answer is to take the folding out of the CV and do this manually. You can generate the indices of the training and testing data using KFold().split(), and iterate over them in this manner:

from sklearn.model_selection import KFold, GridSearchCV
import pandas as pd
from sklearn.ensemble import RandomForestClassifier

iris = pd.read_csv('https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv')

kf = KFold(n_splits=3)

for train_idx, test_idx in kf.split(iris):
    print(train_idx, test_idx)


And what you'll get is three sets of 2 arrays, the first being the indices of the training samples for this fold and the second being the indices of the testing samples for this fold. Using that, you could manually cross-validate like this:

kf = KFold(n_splits=3)

x = iris.drop('species', axis=1)
y = iris.species

max_depths = [5, 10, 15]

scores = []

for i in range(len(max_depths)):
    rfc = RandomForestClassifier(max_depth=max_depths[i])
    scores.append({'max_depth':max_depths[i], 'scores':[]})
    for train_idx, test_idx in kf.split(iris):
        rfc.fit(x.iloc[train_idx], y.iloc[train_idx])
        scores[i]['scores'].append(custom_scorer(y.iloc[test_idx], rfc.predict(x.iloc[test_idx]), train_idx, test_idx)


So that's running once per value in max_depths, setting that parameter to the appropriate value in a RandomForestClassifier. It's then fitting 3 times, once per fold defined in KFold() and passing several things to the call to custom_scorer()...


y.iloc[test_idx] which is our y_true
rfc.predict(x.iloc[test_idx]) which is our y_pred
train_idx which is the indices of the samples of our training data
test_idx which is the indices of the samples of our testing data


Hope that helps. Out of interest: why do you need to know which observations are left out?