Training and Test set
In general, generating independently training and test sets is a legitimate option. The crucial aspect is that both the generating processes are equal. You can check this looking at this example from the author of the caret R package and the Applied Predictive Modeling book. 
However, it is something that can be easily proven with simulations. In what follows, both generating independently training and testing data or splitting the same data in training and testing subsets give the same results. The glm has a median accuracy of 92%.

# simulations with training and test data generating at the same time
n &lt;- 100
accuracy &lt;- vector("numeric")
for (i in 1:1000){
  #create data
  x &lt;- rnorm(n)           # generate X
  z &lt;- 1 + 4*x + rnorm(n) # linear combination with error
  pr &lt;- 1/(1+exp(-z))     # inv-logit function
  y &lt;- pr &gt; 0.5           # 1 (True) if probability &gt; 0.5
  df &lt;-  data.frame(y = y, x = x) 
  train &lt;- sample(x = 1:n, size=n%/%2, replace = F) # sampling training data units
  glm.fit &lt;- glm(y ~ x, data = df[train,]) # fit on the training data
  predicted &lt;- predict.glm(glm.fit, newdata = df[-train,]) # predict on the other data units
  accuracy=c(accuracy, sum(diag(table(predicted&gt;0.5, df[-train,]$y)))/(n%/%2)) # collect accuracy
}
quantile(accuracy, probs = c(0.025, 0.5, 0.975)) # glm accuracy

# simulations with training and test data generating independently
n &lt;- 100 # dataset size
accuracy &lt;- vector("numeric")
for (i in 1:1000){
  #create data
  x &lt;- rnorm(n%/%2)           # generate X
  z &lt;- 1 + 4*x + rnorm(n%/%2) # linear combination with error
  pr &lt;- 1/(1+exp(-z))         # inv-logit function
  y &lt;- pr &gt; 0.5               # 1 (True) if probability &gt; 0.5
  df.train &lt;-  data.frame(y = y, x = x)
  glm.fit &lt;- glm(y ~ x, data = df.train) # fit on the training data
  # generating independent test data
  x &lt;- rnorm(n%/%2)
  z &lt;- 1 + 4*x + rnorm(n%/%2) # linear combination with error
  pr &lt;- 1/(1+exp(-z))         # inv-logit function
  y &lt;- pr &gt; 0.5               # 1 (True) if probability &gt; 0.5
  df.test &lt;-  data.frame(y = y, x = x)
  predicted &lt;- predict.glm(glm.fit, newdata = df.test) # predict on the test data
  accuracy=c(accuracy, sum(diag(table(predicted&gt;0.5, df.test$y)))/(n%/%2)) # collect accuracy
}
quantile(accuracy, probs = c(0.025, 0.5, 0.975)) # glm accuracy