installing spark in this sectioni' going to get you set up using apache sparkand show you some examples of actually using apache spark to solve some of the same problems that we solved using single computer in the past in this book the first thing we need to do is get spark set up on your computer sowe're going to walk you through how to do that in the next couple of sections it' pretty straightforward stuffbut there are few gotchas sodon' just skip these sectionsthere are few things you need to pay special attention to get spark running successfullyespecially on windows system let' get apache spark set up on your systemso you can actually dive in and start playing around with it we're going to be running this just on your own desktop for now butthe same programs that we're going to write in this could be run on an actual hadoop cluster soyou can take these scripts that we're writing and running locally on your desktop in spark standalone modeand actually run them from the master node of an actual hadoop clusterthen let it scale up to the entire power of hadoop cluster and process massive datasets that way even though we're going to set things up to run locally on your own computerkeep in mind that these same concepts will scale up to running on cluster as well installing spark on windows getting spark installed on windows involves several steps that we'll walk you through here ' just going to assume that you're on windows because most people use this book at home we'll talk little bit about dealing with other operating systems in moment if you're already familiar with installing stuff and dealing with environment variables on your computerthen you can just take the following little cheat sheet and go off and do it if you're not so familiar with windows internalsi will walk you through it one step at time in the upcoming sections here are the quick steps for those windows pros install jdkyou need to first install jdkthat' java development kit you can just go to sun' website and download that and install it if you need to we need the jdk becauseeven though we're going to be developing in python during this coursethat gets translated under the hood to scala codewhich is what spark is developed in natively andscalain turnruns on top of the java interpreter soin order to run python codeyou need scala systemwhich will be installed by default as part of spark alsowe need javaor more specifically java' interpreterto actually run that scala code it' like technology layer cake install pythonobviously you're going to need pythonbut if you've gotten to this point in the bookyou should already have python environment set uphopefully with enthought canopy sowe can skip this step