Integrate Spark SQL using Pyspark with python interpreter and pandas and Ipython notebook
Check out the hortonworks sandbox. It's a virtual machine with hadoop and all its components - such as spark ad hdfs - installed and configured. In a addition to that, there is a note book called Zeppelin notebook allowing you to write script in python or other languages. 
You're also free to install python libs and access them through the notebook, even though i'm pretty it comes with it's own data visualisation. 
Note that the spark dataframe type is not compatible with the pandas one. you'll have to convert your data to a simple matrix and integrate back to spark or pandas type.
I want to know the which interpreter is good for Python to use features like Numpy, pandas and matplotlib with the feature of integrated Ipython note book.
Also I want to integrate this with Apache Spark. Is it possible? 
My aim is I need to load different tables from different sources like Oracle, MS SQL, and HDFS files and need to transform them using Pyspark, SparkSQL. And then I want to use the pandas/matplolib for manipulation and visualization.