Is an app on Mac able to know my serial number
iOS apps that run on the device itself are sandboxed. Software running on an external device is not nearly as limited since it has more direct access to the device.

The reason you can't directly see folders on the device when you connect it to a computer is software - the drivers that let the computer OS recognize the device are set up to not show it as a removable storage device in Finder.

No, you did not jailbreak your phone.

Anyone who has physical access to your Mac can obtain the serial number (as can any application running on it). The appearance of the serial number in a log just means it was logged to your computer. It doesn't actually tell you anything about what was done with it. The serial number is not private information, anyway.