Outlier detection by unsupervised algorithm: Fraud Detection
First of all, start with a subset until you know what you are doing.
There is no use in waiting for hours for a result that doesn't work, or to run out of memory, or to optimize, just to find out it does not work.

Secondly, makes sure your preprocessing is very very well done. Bad preprocessing will hurt your algorithms.

From my experience, one-class SVM does not work well. It assumes all your training data is 'normal' class (no outliers), and this a representative sample of all normal data. Secondly,.you need to tune kernel parameters, but how.can you do so without labeled anomalies?

Instead, I'd try knn outlier detection, LOF and LoOP.

But for these you need to make sure your distance is a very good measure of similarity. If you don't preprocess well, distance does not work, and then nearest-neighbor methods don't work either.