Op-amp based audio amplifier not working
The most obvious problem in the circuit is that the + input of the amplifier is floating; it is not biased to any DC voltage level. You need to convey a reference voltage to this input. (This has to be done through a reasonably large resistor; if you just tie the input to a stiff reference voltage, the input will have no impedance to work against.)

Additional problems stem from the choice of part for the circuit. Firstly, although the LM358 can run on as little as 3V, you neglected to look at the common mode input range: according to the data sheet that range is from \$0\$ to \$V^+ - 2\$ or \$0\$ to \$V^+ - 1.5\$: depending of which of two conflicting tables you believe. This means that on such a low supply voltage, you are better of building an inverting amplifier which keeps the common mode input voltages fixed. If you stick to the \$0\$ to \$V^+ - 2\$ recommendation, this means you would build the inverting stage such that it is biased to 1V.

Next, look at the terrible output voltage swing figures! On nice, big 30V supply, the op-amp will only swing up to 26V: a bad sign: the op-amp struggles to come within 4V of the upper power rail. No VOH (high output voltage swing) figures are given for any lower voltage, but for 3V, it will probably be next to nonexistent.

So the fact that the op-amp can "work" on 3V does not mean much: it has poor output voltage swing, and a crummy common mode input range.  If you need an op-amp that works meaningfully on 3V, you need something more modern: something with rail-to-rail output (if not input, too).

Lastly, the LM358 is not very suitable for driving headphones (even when adequately juiced); that application requires an amplifier with an output stage that has decent current driving ability: a power amplifier. (Among the operational amplifiers, one example of a part that is acceptable for driving headphones is the the NJM4556; however it is not low power, and will not run on 3V.)