What machine learning techniques work on ASTs?
Your problem can be solved by existing machine learning methods. So, instead of searching for new technique you should choose from existing techniques. The main part of your problem which separates it from general machine learning problems is AST data which will act as input to your algorithm.

There are a lot of feature extraction techniques available which will help you convert AST data into usable format. I would suggest doing tree traversals to generate a feature vector from your AST and then applying machine learning techniques on it.