Does chilli get milder with cooking?
You'll sometimes hear television cooking show hosts describe what they're doing as "cooking out" the paste. What they're actually doing is altering the flavor of the chili itself, not manipulating the level of capsaicin that was introduced. If you take a typical chili paste which has been combined with garlic and other things and then saute it in a fat, you will lower the level of heat that's tasted (and possibly take some bitterness out, depending on the chili and what's in the paste), but not lower the level that's actually ingested.

If you literally want to cook it out, put some sliced chili with nothing else in a searing hot pan, but make sure no one else is home or going to be home for a while, and wear a respirator. The capsaicin has nowhere to go and vaporizes as it burns (seriously, don't do that, if you did that in my kitchen I'd throw a live crab at you).

Various kinds of fat bond well with the capsaicin, essentially creating a sort of encasement. To demonstrate how this works, grab some sriracha and some cream. Mix in about half a tablespoon of sriracha to about 300 ml of cream, taste it. Now, simmer the cream a bit (about 3 - 5 minutes), take it off the heat, let it rest a few minutes and taste it again. The level of sriracha didn't change, what happened is the cream got its hands on the capsaicin that it contained. It'll continue to get milder the longer it sits. That's why a glass of milk can rescue you from atomic hot wings and why water just broadens the pain as it makes sure your whole palate gets a chance to taste it :) Beer just makes you care less about it.

Now, sugar has the remarkable quality of helping to hide heat with its mighty right of first refusal when it comes to your palate. The heat from the chili shows up and your palate says "what? I'm too busy with this sweet, I can only listen to a little of you". To see how that works, repeat the experiment again, but this time throw a teaspoon or so of confectioner's (or castor) sugar into the cream with the sriracha, along with an additional squirt or two (around 10 ml) of sriracha. That's why coconut milk works so perfectly in a curry. 

The best bet for your recipe is finding a way to get some kind of fat into it. That could be coconut milk, fatty bacon, or whatever makes sense based on how you want to serve it. Then, optionally, sweeten it up a bit (also remember that citrus helps offset sweet, handy to keep that in mind when you're 'fixing' a dish) - some raisins sound like they'd work rather well with what you're doing. Maybe go with coconut milk and raisins, and thin it out with a bit of chicken stock if needed? 

I really try to encourage people to take an hour and just experiment with ways you can alter and sort of control chili, because it is a lovely thing to cook with. Just remember, whatever you put in the pan initially ends up being eaten, you're just hiding a bit of that as you cook.