Using Machine Learning to play the game SET I recently tried to make a CNN which could play the game Set: https://en.wikipedia.org/wiki/Set_(game) 

However, I quickly learned after using my own convolutional neural network and trying to retrain Inception-V3 and AlexNet to classify pictures of three cards into "set" and "notSet", that a CNN is not useful for this task.

Which machine learning algorithm would be useful for playing this game? Would an SVM work?

Here are links to my last question (with some code) and the Github repo which includes sample images of the dataset.

Keras val_acc unchanging when training (same label assigned to all images)

https://github.com/Keyan-Rhm/MLWithSet
So you would like to play this game using images?

If that's the case, I suggest dividing your problem into 2 sub-problems: Image classification (supervised learning) and reinforcement learning.

The first sub-problem will detect the right card and the second sub-problem takes that as an input.

When it comes to game, it's definitely worth trying reinforcement learning. You will only need to create a reinforcement learning environment for this game and use some off-the-shelf algorithms (like OpenAI baselines).