# Ensure the right version of Tensorflow is installed.
!pip freeze | grep tensorflow==2.1

# TODO: change these to reflect your environment
BUCKET = 'cloud-training-demos-ml'
PROJECT = 'cloud-training-demos'
REGION = 'us-central1'

import os
os.environ['BUCKET'] = BUCKET
os.environ['PROJECT'] = PROJECT
os.environ['REGION'] = REGION

%%bash
if ! gsutil ls | grep -q gs://${BUCKET}/; then
  gsutil mb -l ${REGION} gs://${BUCKET}
fi

"""
Explanation: <h1> 2. Creating a sampled dataset </h1>

In this notebook, you will implement:
<ol>
<li> Sampling a BigQuery dataset to create datasets for ML
<li> Preprocessing with Pandas
</ol>
End of explanation
"""


# TODO

"""
Explanation: <h2> Create ML dataset by sampling using BigQuery </h2>
<p>
Sample the BigQuery table publicdata.samples.natality to create a smaller dataset of approximately 10,000 training and 3,000 evaluation records.  Restrict your samples to data after the year 2000.
</p>
End of explanation
"""


## TODO

"""
Explanation: Preprocess data using Pandas
Carry out the following preprocessing operations:

Add extra rows to simulate the lack of ultrasound. 
Change the plurality column to be one of the following strings:

<pre>
['Single(1)', 'Twins(2)', 'Triplets(3)', 'Quadruplets(4)', 'Quintuplets(5)']
</pre>

Remove rows where any of the important numeric fields are missing.
End of explanation
"""


traindf.to_csv('train.csv', index=False, header=False)
evaldf.to_csv('eval.csv', index=False, header=False)

%%bash
wc -l *.csv
head *.csv
tail *.csv

"""
Explanation: <h2> Write out </h2>
<p>
In the final versions, we want to read from files, not Pandas dataframes. So, write the Pandas dataframes out as CSV files. 
Using CSV files gives us the advantage of shuffling during read. This is important for distributed training because some workers might be slower than others, and shuffling the data helps prevent the same data from being assigned to the slow workers.

Modify this code appropriately (i.e. change the name of the Pandas dataframe to reflect your variable names)
End of explanation
"""


