I duplicate a file in photos app mac and it references the same file
Duplicating it simply duplicates the pointer within the library, it does not actually duplicate the underlying file. This occurs whether or not you have a referenced library. You can test this by using the Edit tools to make a change - such as cropping - then duplicating the image. The duplicated image has a "Revert to Original" button in Edit mode that takes you back to what the image looked like before the change, even though this "duplicate" image didn't even exist at that point in time.

Based on this, it appears the duplicate function is intended to allow you to maintain multiple versions of an image at the same time, but still allow you to do so non-destructively.

If you want to physically duplicate the file, you need to do so in Finder and then import that file into Photos.

@Tetsujin is correct about the hard link.