Selection of differential expressed genes I'm working with RNA-seq data. I have 40 tumor samples and 5 Normal samples. Differential analysis with Deseq2 based on Fold change > 1.2 and alpha &lt; 0.05 gave very low number of differentially expressed genes. Only 2 upregulated genes. 

res &lt;- results(dds, lfcThreshold = log2(1.2), alpha = 0.05)


To get more number of differential expressed genes I have few questions now 


Instead of FDR &lt; 0.05 can I use FDR &lt; 0.1 (or) FDR &lt; 0.5. Will there be any low confidence with this?
Can I select differential expressed genes only based on FDR &lt; 0.05 without any fold change cutoff?
As I get very low number of DEGs with FDR &lt; 0.05 &amp; Foldchange > 1.2, Can I select DEGs based on Foldchange > 1.2 and p.value &lt; 0.01 or 0.05 ?


PCA of the samples looks like this:
You can usually get away with FDR &lt; 0.1, but that's as high as you can go. This all presumes you're doing follow-up experiments of some sort, of course. I guess you could increase the FDR more, but you're then really increasing the odds that your follow-up experiments will fail. Obviously increasing the FDR will decrease the confidence in the results, it's called "false discovery rate" for a reason.
Yes, fold-change cut-offs are mostly useful when you need to prioritize hits further. Small changes are almost impossible to validate with less-sensitive methods (qPCR) and less likely to be biologically relevant.
You're just going to waste your time if you ignore FDR.


Remember that all thresholds are intended to increase the odds of success in follow-up experiments. If you have infinite time and resources then you can set these to whatever you want, but you have neither so you better prioritize reasonably.

It's likely that you have a large batch effect in your data, effect sizes in cancer tend to be much larger than this.