The term used to refer to the Upper New York Bay and the term is also used to describe the Port of New York and New Jersey , the port district for New York-Newark metropolitan area , under the jurisdiction of the Port Authority . Originally used to refer to the Upper New York Bay , the term is also used to describe the Port of New York and New Jersey , the port district for New York-Newark metropolitan area , under the jurisdiction of the Port Authority .