How do I setup a server in the cloud for machine learning?
If you are looking for an external storage, then I would suggest you Redshift.

Redshift is a central warehouse and a columnar data store. It allows complex and huge data aggregations and joins; so it is a nice bet for serious Kaggle participants. (I personally use Redshift as data science architecture for kaggle.)

Combinations from some other vendor?

No, a combination of Redshift and S3 is enough. The data is stored in S3, and then loaded into Redshift using the COPY command.

Is there any way to interact with a GUI rather than a command line to set this up?

Yes, there is a CLI interface for Amazon AWS. However, you can use the boto library on Python for handling this stuff.

Would I have to install Anaconda or some other Python distribution before getting started?

Not a compulsion.  Anaconda is a wrapper of numerical and scientific libraries. You can install them on your own or intall Anaconda.

Check out PySpark, which allows you to do handle BigData in Python.