# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/Jeffrowetull/DS-Unit-1-Sprint-1-Dealing-With-Data/blob/master/DS_Unit_1_Sprint_Challenge_1.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + [markdown] id="NooAiTdnafkz" colab_type="text"
# # Data Science Unit 1 Sprint Challenge 1
#
# ## Loading, cleaning, visualizing, and analyzing data
#
# In this sprint challenge you will look at a dataset of the survival of patients who underwent surgery for breast cancer.
#
# http://archive.ics.uci.edu/ml/datasets/Haberman%27s+Survival
#
# Data Set Information:
# The dataset contains cases from a study that was conducted between 1958 and 1970 at the University of Chicago's Billings Hospital on the survival of patients who had undergone surgery for breast cancer.
#
# Attribute Information:
# 1. Age of patient at time of operation (numerical)
# 2. Patient's year of operation (year - 1900, numerical)
# 3. Number of positive axillary nodes detected (numerical)
# 4. Survival status (class attribute)
# -- 1 = the patient survived 5 years or longer
# -- 2 = the patient died within 5 year
#
# Sprint challenges are evaluated based on satisfactory completion of each part. It is suggested you work through it in order, getting each aspect reasonably working, before trying to deeply explore, iterate, or refine any given step. Once you get to the end, if you want to go back and improve things, go for it!

# + [markdown] id="5wch6ksCbJtZ" colab_type="text"
# ## Part 1 - Load and validate the data
#
# - Load the data as a `pandas` data frame.
# - Validate that it has the appropriate number of observations (you can check the raw file, and also read the dataset description from UCI).
# - Validate that you have no missing values.
# - Add informative names to the features.
# - The survival variable is encoded as 1 for surviving >5 years and 2 for not - change this to be 0 for not surviving and 1 for surviving >5 years (0/1 is a more traditional encoding of binary variables)
#
# At the end, print the first five rows of the dataset to demonstrate the above.

# + id="287TpoGKFRVK" colab_type="code" colab={}
#loading data, please standby.....
haberman_data_url = 'http://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.data'
# !curl https://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.data


# + id="kO9KUfDA8Ssj" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 204} outputId="88833ff6-1af4-492e-f937-76e35573c539"
import pandas as pd
#Informative feature names
names = ['age', 'op year', 'nodes', 'survived' ] 
haberman_data = pd.read_csv(haberman_data_url, header=None, names = names)
hd = haberman_data #Shortening the name because it's tedious
haberman_data.shape #(306, 4) Correct number of observations
haberman_data.isnull().sum() #No null values
haberman_data['survived'].replace(2,0, inplace = True) #Replace the twos with zeros
haberman_data.head()



# + [markdown] id="G7rLytbrO38L" colab_type="text"
# ## Part 2 - Examine the distribution and relationships of the features
#
# Explore the data - create at least *2* tables (can be summary statistics or crosstabulations) and *2* plots illustrating the nature of the data.
#
# This is open-ended, so to remind - first *complete* this task as a baseline, then go on to the remaining sections, and *then* as time allows revisit and explore further.
#
# Hint - you may need to bin some variables depending on your chosen tables/plots.

# + id="3E65wFVdUTdg" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 297} outputId="061a242b-4ac5-411a-f7d8-ff39ee0191a2"
hd.describe()

# + id="IAkllgCIFVj0" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 142} outputId="03398416-924c-4776-9120-0e4767a91131"
# Crosstab of survival against age
age_bins = pd.cut(hd['age'], 10)
pd.crosstab(hd['survived'],age_bins, normalize = 'columns')
# No apparent correlation between age and survival

# + id="Xw5kzBbUbXFU" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 159} outputId="e0f211ab-3ab9-4977-fff0-ecaa6b9dd87b"
# How about nodes and survival
node_bins = pd.cut(hd['nodes'],10)
pd.crosstab(hd['survived'], node_bins)
# Other than the majority of people had less than 5.2 nodes, 
# I don't see anything terribly important

# + id="xtl8wd9wbRW2" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 142} outputId="84e4a7b4-9a07-4f2e-fba1-b45484fe2c2c"
# Year vs survival
pd.crosstab(hd['survived'], hd['op year'])
# Starts at half dying, goes down, goes back up and then down etc.
# Nothing here either?

# + id="LVNuO9qGGL7o" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 632} outputId="56fd576f-08d5-448a-8835-9411fdadcffe"
#ok lets try graphing
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

sns.FacetGrid(hd, hue ='survived', size = 8).map(sns.distplot, 'age').add_legend()
plt.show()

# + id="aPjQFdb4I3PZ" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 632} outputId="fe37e2c8-b50d-4af8-98d2-167ee77f6f26"

sns.FacetGrid(hd, hue ='survived', size = 8).map(sns.distplot, 'nodes').add_legend()
plt.show()


# + [markdown] id="ymrWF937UO1s" colab_type="text"
#

# + id="UttK5pSLMZa7" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 632} outputId="7fef5aed-315a-4464-b3cf-80351da2ee2e"
sns.FacetGrid(hd, hue ='survived', size = 8).map(sns.distplot, 'op year').add_legend()
plt.show()
  

# + id="XaaWWy4NWnfO" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 505} outputId="d895dbb5-9784-438a-e61f-026ed2970eb1"


sns.jointplot(hd['age'],hd['nodes'], kind='hex', color='blue')


# + [markdown] id="ZM8JckA2bgnp" colab_type="text"
# ## Part 3 - Analysis and Interpretation
#
# Now that you've looked at the data, answer the following questions:
#
# - What is at least one feature that looks to have a positive relationship with survival?
# - What is at least one feature that looks to have a negative relationship with survival?
# - How are those two features related with each other, and what might that mean?
#
# Answer with text, but feel free to intersperse example code/results or refer to it from earlier.

# + [markdown] id="ktY3x5XkU6f8" colab_type="text"
# One feature that has a positive relationship with survival is age. In my first chart, the survival rate for patients aged thirty to forty is much higher than the rest of the patients. This is shown in the first KDE plot.
#
# The number of nodes has a definite negative relationship with survival. This is clearly seen as the number of nodes increase past five. This is shown in the second KDE plot.
#
# I don't clearly see how age and number of nodes are related, but I can say that the number of nodes drops off at around age seventy as shown in the hexbin plot.

StarcoderdataJupyterScriptsDedupFiltered