Altium - Linking Sheet Entries/Ports with Net Labels
According to Altium documentation, Off Sheet-connectors are only useful for connecting between multiple child-sheets of a given parent page (https://www.altium.com/documentation/altium-designer/sch-obj-crosssheetconnectoroff-sheet-connector-ad). To me, that reads like it only works with hierarchical projects.

Also, you can't connect a Net Label with an Off Sheet-connector, only another Off Sheet-connector.

If you are using a flat project structure, you should use either Net Labels or Ports. If you are using a hierarchical structure, you need (or ought?) to add your sheet entries to a top level schematic (and also use Sheet Entries on both sheets).

An advantage of using ports is that you can add an automatic label next to each port that shows where it connects to (Reports -> Port Cross Reference)