How can I do a literature review efficiently?
If you know that the relevant literature is mostly in one community, then the approach you've described works fairly well. It may be "inefficient" if there are lots of related papers, but (to use computer science jargon), it's efficient in the size of the output :)

I have found that finding a recent survey helps a lot, because it taxonomizes the literature and provides many backward pointers (as well as forward pointers via papers that cite it). 

But ultimately, the best test is if you start coming across the same papers in references over and over again. At that point you can feel a little more confident that you're converging. 

Some specific tips:


Look carefully at how papers are cited. That provides clues on how to prioritize your searching. 
Learn how to skim a paper really quickly to see its main contribution, which will indicate whether it merits further interest. 
If certain researchers keep popping up, go to their websites to see if other papers might be relevant (or if they have a review article that Google search didn't reveal)
Look at venues where papers are typically getting published, and look at recent issues of those journals/conferences to see if you've missed something.


Apart from that, going across communities is trickier, and often requires some luck, or the right keywords. Again, some knowledge of the researchers in the area helps: odds are that if the topic spans disciplines, at least some of the researchers involved also span disciplines and can point (via their work) to new paths to explore (see the above note about conferences)