column must appear in the GROUP BY clause or be used in an aggregate function
You just need to remove the parentheses around the columns in group by (col1, col2). This works in version 9.4 and previous as well:

delete from mytable  
where (col1, col2) in  (
    select col1, col2 from mytable  
    group by col1, col2                   -- &lt;-- changed
    having  count(distinct col3) &gt;1) ;


The reason that it fails (I think) is that while (col1, col2) is equivalent to row(col1, col2), there was some inconsistency in how it was handled in the various clauses which was fixed in 9.5. In previous versions, you could use a more complex construction in WHERE: WHERE (SELECT (col1, col2)) IN .... So this should work in 9.3 as well:

delete from mytable
where (select (col1, col2)) in  (
    select (col1, col2) from mytable  
    group by (col1, col2) having  count(distinct col3) &gt;1) ;