finish feels like sand is in it
Oak is famous for grain raise when varnished; you need to fine sand between coats.

Also important but under-advertised is the importance of applying coats that are appropriately thick when using water-based poly, too thin and the raised grain pokes right through.