In May 1915 he was promoted to the rank of rear admiral and became the Navy's first Chief of Naval Operations (CNO), functionally replacing Rear Admiral Bradley A. Fiske, the last Aide for Naval Operations.
Benson retired from the naval service in September 1919.