How to make Time Machine backups over network without OS X server?
The officially sanctioned answer: buy an Apple Time Capsule device.

Unsupported but well-known alternative: enable a hidden option that allows you to use any network drive. Open the Terminal application and run this one command:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

It will allow you to select any network drive from System Preferences: Time Machine. 

The network drive should be a HFS+ file system over AFP (standard file sharing from another Mac). I would strongly discourage you from using any other setup if you are not prepared to manually restore one file at a time from the backup.