If someone bought an old computer and found Bitcoins in it, could they use it (legally)? Let's say someone bought a used computers which has a copy of a bunch of Bitcoin keypairs. Is he legally allowed to transfer them?

Is this case analogous to finding a pile of cash inside a used bag?
This is going to depend a lot on many surrounding facts. Including where this happens. How you acquired the computer and whether you know who the owner is.

I'm most familiar with the law in Australia (Victoria) where stealing is defined as follows: "A person steals if he dishonestly appropriates property belonging to another with the intention of permanently depriving the other of it." I imagine many common law countries have relatively similar definitions (with important differences for edge cases, of which this is one).

If you see bitcoins as property then your example involves theft. I think most jurisdictions will define bitcoins as a form of property, if they don't there would be all sorts of tax issues. This would be the case in Australia and it would be theft.

To me I don't think this is analogous to finding a pile of cash inside a used bag. I think it is closer to find someone's internet banking username and password inside a bag you purchased. Bitcoins are not physical and the private keys (which is what you would find) merely grant you access to them. Most people would agree that doing anything with the internet banking details would be dishonest.

The difference in my example would be whether destroying the keys would be a crime. It is unlikely to be a problem with internet banking details as there is no expectation this would permanently deprive the person of their property. With bitcoin you don't know if there is a backup so you would at least be recklessly dealing with someone's property.