Remove pages from Pages
If you delete everything on the second page, the second page should disappear. Maybe the template needs to be removed or there are spaces on page 2 that you don't see.

To avoid a lot of guesswork, the function "View -> Show invisibles" should make obvious what is on the page:



Invisible characters will then show on the page: