What the key minis for a new Ultramarine player to get?
Your obvious next steps are a copy of the Codex: Space Marines book and another squad of Tactical Marines.
You're going to need to book to effectively plan your army, and you will find it impossible to win games under the 5th edition rules without enough regular tactical marines around to hold objectives!

From there you'll need to make some key decisions in what type of army you want to have.
Drop Pod Army?
Mechanized (everyone in Rhinos and maybe a land raider for your terminators) Army?
Fast Assault Army with lots of jump pack marines and bike marines, land speeders?

An army that has a focus is usually much more effective in 40K than having one of each different unit type in the book.  The nature of space marines and the need to have tactical squads around means you can devote yourself to a focus that you find appealing without worrying too much about your army becoming entirely one-dimensional.  An upside for marines!

Of course your budget can also help guide early decisions.  An entire army of dudes riding in $40 drop pods adds up fast!  Having the codex will make it easier for you to see what different paths you can follow when making a plan to buy your army.