Installation in system/app without root I know I can install my application in the /system/app folder with root access. But is there any way to do it without root access?
No.

Users are not allowed to write to the /system folder/partition. Any sort of exploit or method that would allow this partition to be writable is equivalent to root access on the device.

Some other examples besides root are: "Rooted recovery" which is a recovery partition that has more features than the standard Android recovery; an "Engineering" (HTC) or Unlocked (everyone else) bootloader; or a file-permissions exploit, which at that point you might as well put su and SuperUser.apk on the phone anyway...