<h4 dir="auto">Describe the workflow you want to enable</h4>
<p dir="auto">I want to be able to use it like this:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="X_train, X_test, X_val, y_train, y_test, y_val = train_test_split(X, y, test_size=0.2, val_size=0.1)"><pre class="notranslate"><span class="pl-v">X_train</span>, <span class="pl-v">X_test</span>, <span class="pl-v">X_val</span>, <span class="pl-s1">y_train</span>, <span class="pl-s1">y_test</span>, <span class="pl-s1">y_val</span> <span class="pl-c1">=</span> <span class="pl-en">train_test_split</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">test_size</span><span class="pl-c1">=</span><span class="pl-c1">0.2</span>, <span class="pl-s1">val_size</span><span class="pl-c1">=</span><span class="pl-c1">0.1</span>)</pre></div>
<p dir="auto">I cannot do this:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.1)"><pre class="notranslate"><span class="pl-v">X_train</span>, <span class="pl-v">X_test</span>, <span class="pl-s1">y_train</span>, <span class="pl-s1">y_test</span> <span class="pl-c1">=</span> <span class="pl-en">train_test_split</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>, <span class="pl-s1">test_size</span><span class="pl-c1">=</span><span class="pl-c1">0.2</span>)
<span class="pl-v">X_train</span>, <span class="pl-v">X_val</span>, <span class="pl-s1">y_train</span>, <span class="pl-s1">y_val</span> <span class="pl-c1">=</span> <span class="pl-en">train_test_split</span>(<span class="pl-v">X_train</span>, <span class="pl-s1">y_train</span>, <span class="pl-s1">test_size</span><span class="pl-c1">=</span><span class="pl-c1">0.1</span>)</pre></div>
<p dir="auto">because the size of validation set will be 0.8 * 0.1, right?</p>
<p dir="auto">Asking how to do a threefold split is the top sklearn question on stackoverflow:<br>
<a href="https://stackoverflow.com/questions/tagged/scikit-learn?sort=frequent&amp;pageSize=50" rel="nofollow">https://stackoverflow.com/questions/tagged/scikit-learn?sort=frequent&amp;pageSize=50</a></p>
<p dir="auto">We have discussed this before but I think this is a good reason to add it - the other option would be to document more explictly doing</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from sklearn.model_selection import train_test_split

X_trainval, X_test, y_trainval, y_test = train_test_split(X, y)
X_train, X_val, y_train, y_val = train_test_split(X_trainval, y_trainval)"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">model_selection</span> <span class="pl-k">import</span> <span class="pl-s1">train_test_split</span>

<span class="pl-v">X_trainval</span>, <span class="pl-v">X_test</span>, <span class="pl-s1">y_trainval</span>, <span class="pl-s1">y_test</span> <span class="pl-c1">=</span> <span class="pl-en">train_test_split</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>)
<span class="pl-v">X_train</span>, <span class="pl-v">X_val</span>, <span class="pl-s1">y_train</span>, <span class="pl-s1">y_val</span> <span class="pl-c1">=</span> <span class="pl-en">train_test_split</span>(<span class="pl-v">X_trainval</span>, <span class="pl-s1">y_trainval</span>)</pre></div>
<p dir="auto">if that's the idiomatic way to do that.<br>
The "issue" with that way is that it's harder to figure out the ratios.</p>
<p dir="auto">If we want to add a threefold split method, there's three options:</p>
<ul dir="auto">
<li>add another parameter to <code class="notranslate">train_test_split</code> (not sure I like this)</li>
<li>create a new method that mirrors the interface of <code class="notranslate">train_test_split</code></li>
<li>create a new method with a better interface than <code class="notranslate">train_test_split</code></li>
</ul>
<p dir="auto">I kinda prefer the last one but it might be confusing to users. My ideal signature would not have <code class="notranslate">*args</code> and explicitly name X and y so we could stratify by default.<br>
Maybe<br>
<code class="notranslate">naming_is_hard(X=None, y=None, fit_parms=None, train_size=None, val_size=None, test_size=None)</code>.</p>
<p dir="auto">I'm not sure if it would make sense to pass a <code class="notranslate">cv</code> object here? or a CV class that's internally instantiated? Or we could have <code class="notranslate">stratify</code> and <code class="notranslate">grouped</code> options? Maybe the last makes most sense?</p>