Consider a .csv file that contains a set of video names like so:There are happy days,1204923Beware of ignorance,589636Bloody Halls MV,258933Dream Theater - As I Am - Live in...,89526The intent of the code I built is to filter items in the csv depending on the list of excluded items. Therefore, if the name of the video contains a word in the list of excluded items, it'll be rejected for saving. The following is the code:exclude_list = [mv,live,cover,remix,bootleg]data_set = []with open('video_2013-2016.csv', 'rb') as f:    reader = csv.reader(f)    for row in reader:        # Only record videos with at least 100 views        if int(row[1]) > 99:            # A test list that holds whether the regex passes or fails            test_list = []            for ex in exclude_list:                regex = re.compile(.*(+ex+).*)                if regex.search(row[0]):                    test_list.append(False)                else:                    test_list.append(True)            # Depending on the results, see if the row is worthy of saving            if all(result for result in test_list):                data_set.append(row)I know the code I wrote above is quite inefficient, and I've seen examples of list comprehensions that can do a better job, but I do not quite understand how list comprehension can work in this case. I just hate it that I have to create the regex variable many times and it feels like a waste of resource.
Improving CSV filtering with Python using regex
python;performance;regex;csv
The CSV file contains text in some text encoding, and should not be opened in binary mode.You should construct one regular expression to find any of the forbidden words.  It appears that you intended to do a case-insensitive search, but didn't write the code that way.  When constructing the regex, you should escape the strings, in case they contain any regex metacharacters. You don't need .*, since re.search() will look for the pattern anywhere in the string, nor do you need capturing parentheses.If your comment says 100, then your code should have 100 rather than 99.I suggest doing a destructuring assignment title, view_count = row to make it clear what each column represents.with open('video_2013-2016.csv') as f:    forbidden = re.compile('|'.join(re.escape(w) for w in exclude_list), re.I)    for row in csv.reader(f):        # Only record videos with at least 100 views and none of the bad words        title, view_count = row        if int(view_count) >= 100 and not forbidden.search(title):            data_set.append(row)