OpenAI spinning up convolutional networks with PPO
I have written a tutorial on using OpenAI Spinning Up in a image-based PyBullet + Gym environment here

In order to be able to use spinup for an image-based environment I had to fork it here and add CNN to PPO's core.py