Generally speaking, shouldn't GDP roughly increase equivalent to population growth?
Basically the answer is yes

Population growth is a key factor in GDP growth. Population creates both demands for goods and labour supply

Unlike GDP, GDP per Capita, cannot, obviously rely on population growth, and that is (or at least should be) the goal of governments. and unlike population, the "rest of the factors" affect GDP per Capita just the same

A couple of extra points besides the obvious: population growth, by births or by immigration, affect age distribution, which affects both supplies of labour amd demands for goods and services

Because of it, population stagnation may mean decrease in GDP per Capita and long term economic issues