data preparation the third phase in the crisp-dm process takes place after gaining enough knowledge on the business problem and relevant dataset data preparation is mainly set of tasks that are performed to cleanwranglecurateand prepare the data before running any analytical or machine learning methods and building models we will briefly discuss some of the major tasks under the data preparation phase in this section an important point to remember here is that data preparation usually is the most time consuming phase in the data mining lifecycle and often takes to time in the overall project however this phase should be taken very seriously becauselike we have discussed multiple times beforebad data will lead to bad models and poor performance and results data integration the process of data integration is mainly done when we have multiple datasets that we might want to integrate or merge this can be done in two ways appending several datasets by combining themwhich is typically done for datasets having the same attributes merging several datasets together having different attributes or columnsby using common fields like keys data wrangling the process of data wrangling or data munging involves data processingcleaningnormalizationand formatting data in its raw form is rarely consumable by machine learning methods to build models hence we need to process the data based on its formclean underlying errors and inconsistenciesand format it into more consumable formats for ml algorithms following are the main tasks relevant to data wrangling handling missing values (remove rowsimpute missing valueshandling data inconsistencies (delete rowsattributesfix inconsistenciesfixing incorrect metadata and annotations handling ambiguous attribute values curating and formatting data into necessary formats (csvjsonrelationalattribute generation and selection data is comprised of observations or samples (rowsand attributes or features (columnsthe process of attribute generation is also known as feature extraction and engineering in machine learning terminology attribute generation is basically creating new attributes or variables from existing attributes based on some ruleslogicor hypothesis simple example would be creating new numeric variable called age based on two date-time fields--current_date and birth_date--for dataset of employees in an organization there are several techniques with regard to attribute generation that we discuss in future attribute selection is basically selecting subset of features or attributes from the dataset based on parameters like attribute importancequalityrelevancyassumptionsand constraints sometimes even machine learning methods are used to select relevant attributes based on the data this is popularly known as feature selection in machine learning terminology