Face Overlapping Edge
As previously said by Carlo: "Concave polygons can cause artifacts" because Blender is not sure how he is supposed to fill the surface between the vertices.

So here is a better way to make a hole in a 4x4 surface. I think you should avoid Booleans as much as possible, especially for such a simple operation. To make a hole you can simply select the central point and delete it. Then to round the shape, select the edge loop and use the LoopTools addon > Circle. Then you can extrude once, then scale down the extrusion, give it the shape you want, etc...