What's the point of having ports as a territory?
As you deduced, the huge advantage is that it gives you a safe harbour to muster ships into, which is invaluable when all adjacent sea areas are occupied. Relatedly, they also give your ships a safe place to retreat to when your sea areas are attacked, so you don't risk losing routed ships that you would presumably prefer to keep in order to take them back.

I'm assuming you're playing AGoT 2nd Edition here, because in the 1st Edition (unexpansioned), there were no ports, and that became a very real problem in a number of the games that I played, and a huge strategic concern in the rest: Losing control of your sea territories could be devastating, especially for island nations such as Pyke or Dragonstone. Even for cities on the continent, the power of ships &mdash; from support to ship-transport (and obviously preventing your opponents from the same) &mdash; was a major consideration, and if you were by bad luck or worse strategy driven out of the sea entirely and then blocked, you were in very real trouble that could be very difficult if not impossible to recover from.

The introduction of ports with the Clash of Kings expansion (and its default inclusion in the second edition) was, in my opinion, the most important single upgrade to the base game.

As for what else you can and cannot do in a port, your list is somewhat misleading (see page 25 of the rules): Ships in port can support and raid, but only on the adjacent sea area. Even consolidating isn't off the table entirely, it's merely contingent on the adjacent sea area not being under enemy control. Not to mention that your ships can do all this from a territory that's safe from direct attack.

So even if they're not quite as useful as a ship that's actually, y'know, out to sea, they're not really all that useless either.