The former England international, 34, has signed on a free transfer after winning friends on and off the field in his three months at the Ricoh Arena.
"He's great to have around the dressing room," said City manager Tony Mowbray.
"The rest of the squad feed off his experience. I'm sure it'll have a positive impact on our young players."
Cole scored a cracking free-kick in a 4-3 win over Barnsley after arriving on 19 October, but has only made seven appearances in total because of injury niggles.
"Our staff have worked hard to build up his fitness and physicality, and we've seen Joe really buy into what we're trying to do here," added Mowbray.
"I only see his performances for the club getting better. He is a fantastic option to have from now until the end of the season."
Cole began his career at West Ham before moving in 2003 to Chelsea, for whom he made 183 league appearances, scoring 28 goals.
After seven years at Stamford Bridge and two Premier League titles, he moved to Liverpool, but had a loan spell with French Ligue 1 side Lille before returning to boyhood club West Ham in 2013.
He made 31 appearances in a one-year spell at Upton Park, scoring five goals, before switching to Villa, for whom he made four starts.
Cole scored 10 goals in 56 international appearances for England.
Find all the latest football transfers on our dedicated page.
Coventry City's on-loan Aston Villa midfielder Joe Cole has completed a move to the Ricoh Arena on a deal until the end of the season.