steps and data retrieval and preparation data retrieval and data preparation are two distinct steps in the data science processand even though this remains true for the case studywe'll explore both in the same section this way you can avoid setting up local intermedia storage and immediately do data preparation while the data is being retrieved let' look at where we are in the data science process (see figure data science process setting the research goal data retrieval no internal data available internal data retrieving data data ownership no internal data available external data wikipedia figure data science process step data retrieval in this case there' no internal dataall data will be fetched from wikipedia as shown in figure you have two possible sourcesinternal data and external data internal data--you have no disease information lying around if you currently work for pharmaceutical company or hospitalyou might be luckier external data--all you can use for this case is external data you have several possibilitiesbut you'll go with wikipedia when you pull the data from wikipediayou'll need to store it in your local elasticsearch indexbut before you do that you'll need to prepare the data once data has entered the elasticsearch indexit can' be alteredall you can do then is query it look at the data preparation overview in figure as shown in figure there are three distinct categories of data preparation to considerdata cleansing--the data you'll pull from wikipedia can be incomplete or erroneous data entry errors and spelling mistakes are possible--even false information isn' excluded luckilyyou don' need the list of diseases to be exhaustiveand you can handle spelling mistakes at search timemore on that later thanks to the wikipedia python librarythe textual data you'll receive is fairly clean already if you were to scrape it manuallyyou' need to add html cleaningremoving all html tags the truth of the matter is full-text search tends to be fairly robust toward common errors such as incorrect values even if you dumped in html tags on purposethey' be unlikely to influence the resultsthe html tags are too different from normal language to interfere