categorical data free strings that can be semantically mapped to categories structured string data text data categorical data is data that comes from fixed list say you collect data via survey where you ask people their favorite colorwith drop-down menu that allows them to select from "red,"green,"blue,"yellow,"black,"white,"purple,and "pink this will result in dataset with exactly eight different possible valueswhich clearly encode categorical variable you can check whether this is the case for your data by eyeballing it (if you see very many different strings it is unlikely that this is categorical variableand confirm it by computing the unique values over the datasetand possibly histogram over how often each appears you also might want to check whether each variable actually corresponds to category that makes sense for your application maybe halfway through the existence of your surveysomeone found that "blackwas misspelled as "blakand subsequently fixed the survey as resultyour dataset contains both "blakand "black,which correspond to the same semantic meaning and should be consolidated now imagine instead of providing drop-down menuyou provide text field for the users to provide their own favorite colors many people might respond with color name like "blackor "blue others might make typographical errorsuse different spellings like "grayand "grey,or use more evocative and specific names like "midnight blue you will also have some very strange entries some good examples come from the xkcd color surveywhere people had to name colors and came up with names like "velociraptor cloakaand "my dentist' office orange still remember his dandruff slowly wafting into my gaping yaw,which are hard to map to colors automatically (or at allthe responses you can obtain from text field belong to the second category in the listfree strings that can be semantically mapped to categories it will probably be best to encode this data as categorical variablewhere you can select the categories either by using the most common entriesor by defining categories that will capture responses in way that makes sense for your application you might then have some categories for standard colorsmaybe category "multicoloredfor people that gave answers like "green and red stripes,and an "othercategory for things that cannot be encoded otherwise this kind of preprocessing of strings can take lot of manual effort and is not easily automated if you are in position where you can influence data collectionwe highly recommend avoiding manually entered values for concepts that are better captured using categorical variables oftenmanually entered values do not correspond to fixed categoriesbut still have some underlying structurelike addressesnames of places or peopledatestelephone working with text data