If I have so few armies I can't attack any more, do I lose the game?
Most certainly not! 

You will still receive armies and be able to attack in a future turn, as the rules make clear (see page 6: Getting and Placing New Armies - Territories) that every player with at least one territory receives at least three armies each turn for territories, even if owning fewer than 9 territories. 

In order to not be a target of other players while you are weak, it is best to hold as few cards as possible, ideally zero, until you are strong enough to have a chance at defending your self, so don't attack at all.