title: Issue in Spark not able to read S3 subfolders of a hive table
body: <p>I have 3 non-partitioned tables in Hive.</p>
  
  <pre><code>drop table default.test1;
  CREATE EXTERNAL TABLE `default.test1`(                                                                                                    
  `c1` string,                                                                                                                                                                     
  `c2` string,                                                                                                                                                                    
  `c3` string)                                                                                                                                                                
  ROW FORMAT SERDE                                                                                                                                                                        
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'                                                                                                                         
  STORED AS INPUTFORMAT                                                                                                                                                                   
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'                                                                                                                       
  OUTPUTFORMAT                                                                                                                                                                            
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'                                                                                                                      
  LOCATION                                                                                                                                                                                
  's3://bucket_name/dev/sri/sri_test1/';
  
  drop table default.test2;
  CREATE EXTERNAL TABLE `default.test2`(                                                                                                    
  `c1` string,                                                                                                                                                                     
  `c2` string,                                                                                                                                                                    
  `c3` string)                                                                                                                                                                
  ROW FORMAT SERDE                                                                                                                                                                        
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'                                                                                                                         
  STORED AS INPUTFORMAT                                                                                                                                                                   
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'                                                                                                                       
  OUTPUTFORMAT                                                                                                                                                                            
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'                                                                                                                      
  LOCATION                                                                                                                                                                                
  's3://bucket_name/dev/sri/sri_test2/'; 
  
  
  drop table default.test3;
  CREATE EXTERNAL TABLE `default.test3`(                                                                                                    
  `c1` string,                                                                                                                                                                     
  `c2` string,                                                                                                                                                                    
  `c3` string)                                                                                                                                                                
  ROW FORMAT SERDE                                                                                                                                                                        
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'                                                                                                                         
  STORED AS INPUTFORMAT                                                                                                                                                                   
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'                                                                                                                       
  OUTPUTFORMAT                                                                                                                                                                            
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'                                                                                                                      
  LOCATION                                                                                                                                                                                
  's3://bucket_name/dev/sri/sri_test3/';`
  
  `
  --INSERT:
  insert into default.test1 values("a","b","c");
  insert into default.test2 values("d","e","f");
  insert overwrite default.test3 select * from default.test1 union all default.test2;
  `
  </code></pre>
  
  <p>If I look in s3 two subfolders have been created because of the union all operation. </p>
  
  <pre><code>aws s3 ls s3://bucket_name/dev/sri/sri_test3/`;
  
  PRE 1/
  PRE 2/
  </code></pre>
  
  <p>Now the issue is that if I try to read the default.test3 table in pyspark and create dataframe.</p>
  
  <pre><code>df = spark.sql("select * from default.test3")                               
  df.count()                                                                  
  0  
  </code></pre>
  
  <p>How to fix this issue?</p>