"a" or "an" with big O notation?
Evidence for "an":


Ajtai, Miklós, János Komlós, and Endre Szemerédi. "An O(n log n) sorting network." Proceedings of the fifteenth annual ACM symposium on Theory of computing. ACM, 1983.
Asadpour, Arash, et al. "An O(log n/log log n)-approximation Algorithm for the Asymmetric Traveling Salesman Problem." SODA. Vol. 10. 2010.
Bracha, Gabriel. "An O(log n) expected rounds randomized Byzantine generals protocol." Journal of the ACM (JACM) 34.4 (1987): 910-920.
Hwang, F. K. "An O(n log n) algorithm for suboptimal rectilinear Steiner trees." Circuits and Systems, IEEE Transactions on 26.1 (1979): 75-77.
Tarjan, Robert E., and Christopher J. Van Wyk. "An O(n log log n)-time algorithm for triangulating a simple polygon." SIAM Journal on Computing 17.1 (1988): 143-178.
Yao, Andrew Chi-Chih. "An O (|E| log log|V|) algorithm for finding minimum spanning trees." Information Processing Letters 4.1 (1975): 21-23.


Of course, these examples alone merely suggest that "an" is acceptable. This list does not show that "an" is necessarily standard or preferred nor does it show that "a" is unacceptable.