Skewed two class data set
It is typically called a class imbalance issue, where the occurrence of a label happens so infrequently that makes predictions unreliable.

For instance, if I know Vancouver, Canada rains 85% of the time in winter, I would simply predict that it is raining when it is winter + vancouver. You don't want your algorithm to favour one label over another because one label predominates.

One common strategy would be resampling. If you have enough data, downsampling could make more sense as (oversampling requires the creation of synthetic data (e.g., SMOTE, etc.)). so that the algorithm can properly learn the difference between the two classes and wouldn't favour one over another. 50-50 split between the two classes would probably be a good starting point, but it also depends on what is available. You still want your negative labels to be representative, and even an 80-20 split would be a vast improvement already.

Another common solution is to increase the penalty of incorrect predictions. The way to think about it, false positive = admin spending time investigating a false alarm; false negative = rouge activities went undetected. For different business, one cost could be more severe than another so you could potentially say getting a false negative is 1000x more costly to business, etc.

Not knowing your data, those are probably the first two things (separately or together) to try. Most ML packages could handle either strategy rather easily so that's why I think those are reasonable things to try.

There are many thorough articles and tutorials with additional strategies available. Try the keywords anomaly detection and class imbalance and it seems to give me some pretty good results.