Training models from sklearn using tf.distribute.MirroredStrategy
There are a few approaches that allow you to do basic ML modelling using a GPU.

First of all, in the code as you presented it, the tensorflow MirroredStrategy unfortunately has no effect. It will only work with tensorflow models themselves, not those from sklearn. In fact, sklearn does not offer any GPU support at all.

1. CUML

An Nvidia library that provides some basic ML model types and other things, often offering the exact same API (classes and functions) as SciKit-Learn. Coming from Nvidia, this of course means everything is built with the GPU in mind.

This is part of the larger RAPIDS toolset (incubated by Nvidia). Maybe there are other tools there that can be helpful, like their XGBoost library.

2. Tensorflow / PyTorch + NumPy

These framework are not just for complicated Deep Learning, you can really use them to perform any basic modelling and leverage their GPU support. Their documentation contains examples, otherwise something like Hands On Machine Learning (a book with an accompanying set of Jupyter notebooks) is a nice way to dig in.

These frameworks work well with the normal scientific stack in Python (such as NumPy, Scipy, Pandas) because numpy arrays and the frameworks' Tensor objects are plug-and-play for most cases.



3. Another option:

Stick to sklearn while you are learning about the models, how they work and so on. If you want to just do anything with the goal of learning about about GPU usage, the two options above are the most modern ways to get started.