Function to perform checks on data I'm currently trying to do data quality checks in python primarily using Pandas.
The code below is fit for purpose. The idea is to do the checks in Python and export to an Excel file, for audit reasons.
I perform checks like the below in 4 different files, sometimes checking if entries in one are also in the other, etc.
df_symy is just another DataFrame that I imported inside the same class.

I'm wondering if there's a better way to do this task, for example:

create a function for each check
create a more meaningful way to check the data and present the results, etc


def run_sharepoint_check(self) -&gt; pd.DataFrame:
        &quot;&quot;&quot;Perform checks on data extracted from SharePoint.

        Check 1: Amount decided is 0 or NaN
        Check 2: Invalid BUS ID
        Check 3: Duplicated entries. It also tags the latest entry
        Check 4: SYMY Special not in SharePoint
        &quot;&quot;&quot;
        # Use this function to check the data we have from SharePoint
        # and return DF with checks in the excel file 
        # Another function will be used to process the data and produce
        # the final list
        df = self.sharepoint_data.copy()

        df_symy = self.symy_data
        df_symy = df_symy.loc[df_symy['COMMITMENT_TYPE'].str[0] == 'S']
        
        symy_cols = ['BUYER_NUMBER', 'Balloon ID',
                     'CLD_TOTAL_AMOUNT', 'POLICY_CURRENCY']
        
        df = df.merge(right=df_symy[symy_cols],
                      how='outer',
                      left_on=['Balloon ID', 'BUS ID'],
                      right_on=['Balloon ID', 'BUYER_NUMBER'])
        
        check_1 = df['Amount decided'] == 0 | df['Amount decided'].isna()
        df.loc[check_1, 'check_1'] = 'Amount decided is 0 or NaN'

        check_2 = df['BUS ID'].isna()
        df.loc[check_2, 'check_2'] = 'Invalid BUS ID'

        check_3 = df.duplicated(subset=['Balloon ID', 'BUS ID'], keep=False)
        df.loc[check_3, 'check_3'] = 'Duplicated entry'
        
        check_3_additional = ~df.duplicated(
            subset=['Balloon ID', 'BUS ID'], keep='first'
        )
        # Filter only the entries that are duplicated
        # Out of those, the first one is the latest
        df.loc[(check_3) &amp; (check_3_additional),
               'check_3'] = 'Duplicated entry (Latest)'

        # Match Balloon+BUSID (SYMY) to SharePoint
        check_4 = (~df.duplicated(subset=['Balloon ID', 'BUS ID'],
                                  keep='first')) &amp; (df['BUS ID'].isna())
        df.loc[check_4, 'check_4'] = 'SYMY SA not in SharePoint'

        check_cols = ['check_1', 'check_2', 'check_3', 'check_4']

        # .fillna('OK') just for visual purposes in Excel.
        df[check_cols] = df[check_cols].fillna('OK')
        
        # self.data_checks_dfs in the init method is an empty dictionary.
        self.data_checks_dfs['SharePoint_checks'] = df
       
        return df

So, how can I improve this?
Does anyone have this type of task that has been automated using Python?
It seems difficult to comment on the meaningfulness of your proposed solution. You are the one to decide on that since you know the context around your problem. With that being said, I'm afraid there is not much to go on here.
Nevertheless, one observation I make about your code is that the checks use .loc to locate rows that receive a &quot;non-OK&quot;, and finally, after all checks, you use a fill to turn nans into &quot;OK&quot;. To make your code more concise and likely faster, consider using np.where. So for instance, your check_1 would be turned into:
df[&quot;check_1&quot;] = np.where(
   (df['Amount decided'] == 0) | (df['Amount decided'].isna()),
   &quot;Amount decided is 0 or NaN&quot;, &quot;OK&quot;)

Also, I would rename check_N into something that actually means a little more, like amount_check for check_1 and so on.