Is a comet necessarily confined to the solar system?
It means that the sum of the kinetic and gravitational potential energies can be greater than zero. Such objects are not bound to the solar system. They fall towards the Sun on hyperbolic trajectories, picking up speed and kinetic energy as they do so. After rounding the Sun they then have enough speed to escape from the solar system again.

The origin of such comets is not known for certain - some may have come from other star systems. Some comets can enter the solar system on parabolic orbits - these have come from a long way out - perhaps the Oort cloud - and are just bound to the solar system. But interactions with other planets and the Sun can result in them gaining energy and leaving the solar system on unbound hyperbolic orbits.