Can you build a retaining wall vertically with no angle?
I have built many vertical retaining walls. Drainage is the first concern then the height. Code may require rails over several feet (in my area 3' is the max). You did not specify the number of steps or the rise (if more than 3' even with drainage usually reinforcements are required). With some walls this is drilled holes that are pinned together. Some are mortar. 

So yes you can go vertical but more information is needed on the height and conditions to give a specific recommendation.