Parameter Adjustment based only on tagged predictions not sure that this is the best place to post this but if not, please let me know if there is a better stack community.

I have an anomaly detection method which has some parameters. I have some data that I need to find anomalies for. This data is not tagged. I use the anomaly detection method on the data using some default parameters. It predicts some anomalies. The user looks ONLY AT PREDICTED ANOMALIES and determines if he or she agrees with them or not (is the predicted anomaly truly an anomaly?) Now I need to adjust the parameters of the anomaly detection method so it can do better next time whether that is on the same data set or a new data set.

More specifically:

1) system predicts some anomalies

2) user looks at those predicted anomalies and says if they agree or disagree with them

3) system looks at those predicted anomalies and whether or not the user agrees with them and makes adjustments to the parameters based on this information.

The ``making adjustments" part is what I am looking for ideas about. As for why I am not listing a specific method, I was hoping for more generic ideas.

How do I go about this (the parameter adjustment) for either situation (same or new data set)?
This is more a comment than an answer but the adjustment part is likely to depend on the model you have. 
So it will be difficult to give you a general method. If your model is a simple rule base one, the adjustment could consist in adding another rule, if it is a neural network then you'll go with backpropagation to adjust the weights of the nodes, if it's a Bayesian tree then you could grow another tree taking into account your new outliers, if it's Bob, your dear colleague, you'll have to make sure that he takes the new info into account next time he classifies something as being an outlier.