Are there any ways to increase dopamine and serotonin levels when working or learning something complex?
I'm guessing you don't want to generally increase the level of those chemicals in the brain, just in the reward-motivation area.
In order to do so, you need to be rewarded and motivated, obviously. The learning need to be exciting,  with feedback and reinforcement. Socializing it will help, too.
There is a relatively new topic, called gameificatin, that try to do exactly that. It's the process of making the learning a game, so we can learn better,based on the reward-motivation effects.
In my opinion, there is no "hacking" the brain. Doing so is using drugs, and there is a price tag for that. If you want to use the normal brain pathways, there are no shortcuts.