Is it an antipattern to avoid API Gateway for certain tasks? I'm creating a platform architecture for my project and I'd like to use an API Gateway of some sort for handling most requests to a Kubernetes cluster (with a few microservices).

There is, however, one task that relies a lot on performance and sheer number of connections (think IoT metrics kind of traffic, a lot of connections sending data often, most likely gRPC). I'd always hit a single service in the cluster. It's also different because it will come always from a single physical device (IoT hub, if we're using that analogy) unlike the rest (being web browser, mobile app, 3rd party access, etc).

Since I'd like to offload the API Gateway from that kind of traffic to provide stability and performance for remaining traffic, would it be an antipattern to have a direct connection for this case? Or maybe there's another solution for it?
You could have a separate API gateway for that kind of traffic.

Check out the "Variation: Backends for frontends" section of the following article:

https://microservices.io/patterns/apigateway.html