problem with 6 bit adder
The way I did the calculations using pen and paper were wrong. 

For sel = 1,

When x = 111111 and y = 100000

y'= 011111

I was adding y' and 1 first and then adding x. This way is wrong.

Binary numbers are added bit by bit. So the sel bit which is 1 has to be added with the Least significant bits first and so on. Add x, y' and sel together. 

  111111
  011111
      +1
  ______
1 011111 ,  Value of Overflow = 1 ex-or 1 = 0 ( Correct ) 


If I add y' and 1 together first then 
  011111
       1
  ______
  100000

  111111
  100000
  ______
1 011111   , Value of Overflow = 1 ex-or 0 = 1 ( Incorrect ) 


Similarly for the other case.