Hypothetical Long-term Business Cycle
You are correct that there have in fact been periods of time when growth was flat, or even negative. For example, if one could observe the economy of the Cahokia over time, one would expect to see a period of negative growth. Further, the rapid growth we're to which we are accustomed is a very recent phenomenon. 

It's impossible to know whether growth will continue at such a pace or slow dramatically; prominent economists such as Robert Gordon have argued that future growth may be much slower (though few argue that it will be flat or negative), a set of hypotheses known as "secular stagnation".

So while your textbook, showing cyclical variation around a positive long-term trend, correctly depicts modern business cycles, there's no reason to believe that such a trend must appear in all societies at all times.