title: Deep Learning : The validation set and test set are the same, but the results are quite different
body: <p>During the training process, I added a validation mode and <strong>used PSNR as an evaluation metric</strong> (<strong>not loss function</strong>). The validation stage <strong>saves the current model parameters</strong> with the largest psnr. However, in the <strong>testing phase</strong>, I load the model parameters and <strong>use the same data as the validation set as the test set</strong>, however, the psnr of the test is <strong>quite different</strong> from the psnr of the validation.
  And, the network model is the Unet without BN and Dropout</p>
  <p><strong>Train Code simply as follows</strong></p>
  <pre class="lang-py prettyprint-override"><code># pytorch framework
  msxpsnr = 0
  for epoch in range(epochs):
       model.train()
       for idx, data in enumerate(trainDataLoader):
            optimizer.zero_grad()
            out = model(data[1])
            loss = criterion(out, data[1])
            loss.backward()
            optimizer.step()
       model.val()
       for idx, data in enumerate(testDataLoader):
            out = model(data[1])
            psnr = compare_psnr(data[0], data[1])
       if psnr &gt; maxpsnr :
            torch.save(model.state_dict())
            msxpsnr = psnr
  </code></pre>
  <p>Simple Example:</p>
  <pre><code>Training Phase Information:
  
  ---Validation----Epoch : 1904 ,  CurrentMaxPSNR : 27.923486654016344 , SSIM : 0.7779479678648872
  
  Test Phase : I load the model1904.pth, result...
  ---Test------PSNR : 27.479514001762144 , ---------SSIM : 0.7835264324520744
  </code></pre>
  <p>As a novice, I really hope to get relevant guidance</p>