Finding matches in Excel files, adding to a Pandas Dataframe
The issue has been resolved.

The problem was I was creating the pattern r = pattern(keys) for each keyword (remind you i had 600 keywords) for each cell of the excel which was completely unnecessary. 

So if i had a 10(rows) * 10 (columns) excel file, the same 600 pattern was getting created 600 * 10 * 10 times.

So I created one dictionary of patterns just once and used the same 10 * 10 times.

for cat,key in keyword_dic.items():
    temp = [pattern(keyword) for keyword in key]
    list_of_patterns[cat] = temp


Thanks everyone, for taking the time to go through the post!