Best practice - memory management when data wrangling
Programming best practices that are specifically usefull for ML / DS :

Know and handle your data types, with a dict for example, and pass this dict as and agrument to your read csv. You can get important memory usage reduction by using the right types of float / int and category instead of string. Float precision is rarely important in ML (and usually achieve better statistical performance trough faster calculations).

As a part of your model building process, it is usefull to do some variable selection (to reduce the number of columns) or sampling (to reduce the number of rows). Sometimes just for testing your pipeline, sometimes to get a baseline, sometimes as a long run solution.

You have to actively avoid multiplying you dataframes. I usually check for that in real time by seeing what memory is consumed. (Either trough command line interface like htop, or trough ad-hoc memory requests in the python environnement). This means you have to look into your functions which are duplicating your df. Then you have some work to do to find an alternative that is low memory (using vectorized calculation, using a 'in_place = True' option ... etc). During this step you also have to check that your operations are not upgrading the data types you specified in the first part.

Clean after yourself. You have to delete the objects you don't use (the raw data after you preprocessed them, the whole data you just splitted ... etc.). Basically using del() everywhere you can and the garbage collector gc.collect() often.