How to fix chipped cement
A small-ish tub of ready mix stucco patch should do the trick. 

A few notes:


be sure to knock off anything loose underneath
check the label for application/curing temperature. It may be too cold to use right now. (I don't know of any product that's specifically designed for cold weather application, though it might exist.)
a coarse sponge might help you get close to the texture, or you may find that you need to add something to the top layer (like coarse sand). It's not absolutely clear what your existing finish is made up of. 


(edit to respond to your edit) I wouldn't use the quikrete you linked. Color is going to be hard to match. If consistency is something you're looking for, you'll probably need to do the entire walls up to the next corner.