How are transactions secured?
First of all, nothing is encrypted in Bitcoin, there is no "encryption" involved whatsoever. Cryptography: yes, but encryption: no.

The main aspect of private vs public key signing, is dat you can openly share your public key (that's what bitcoin balances are associated with), and then using your private key you can create a "digital signature" for a transaction spending coins from that public key. Other people can verify if this signature is correct, thus knowing that the transaction was signed by the one holding the private key. But here's the catch: this verification can be done without having the private key.

So essentially, the digital signature algorithm allows you to prove that you have the private key that corresponds to a certain public key, without sharing the actual private key.

And, obviously, neither is there a feasible way to reconstruct the private key from a given public key and a valid signature.

This is called ECDSA or "elliptic curve digital signature algorithm", because the inner workings are based on elliptic curves.

How this all works technically is somewhat complicated, there's some fancy maths involved. If you want to get into the dirty details, here's a good explanation: http://kakaroto.homelinux.net/2012/01/how-the-ecdsa-algorithm-works/