# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] _uuid="fb2dc4da758ebc6382dcb418330f2daad96dd5c4"
# > > #  retinopathy convNet Case2 (with convNet + dense)
# <NAME> 
# [Helsinki Metropolia University of Applied Sciences](www.metropolia.fi/en)<br>

# + [markdown] _uuid="d77cf99304dd187aa5b3c5a8bbd7809616cf0db3"
# ## Xception convNet and dense, for binary classification retinopathy
#
# Make a convnet architecture and try to classify images to predict diabetic retinopathy in binary fashion (sick or healthy).
#
# I will attempt to use pre-trained Xception convNet, but I will initialize it to random weights and try different training schemes.
#
# Earlier tests I tried to do with underfitting architectures, but ready convNets such as Xception of VGG19 should be powerful enough for the retinopathy classification. You should always try to first get an overfitting model with good trainingaccuracy in my limited experience. That way you can see that the model is overfitting and is powerful enough to learn the trainingset, even if the validationaccuracy is lagging behind.
#
# 1.) maybe and train convLayer parts of the network for a few epochs
#
# 2.) Then, I will freeze that convNet layers, then I will train the dense layers only (???)
#
# 3.) possibly initalize random and train the whole convNet + dense. (easiest to do)
#
#
# ## useful links & resources
# * here was a useful tutorial for imagegenerators and pre-trained convnet https://towardsdatascience.com/keras-transfer-learning-for-beginners-6c9b8b7143e
# * another good help link https://github.com/keras-team/keras/issues/4465
# * Im trying to use pretrained convnet architecture such that not the  entirety of my days on this earth are spent on convNet model training
# * we will see what kind of results we get
# * After having tried most tips, the last one I didn't use (yet)  was to crop the images into larger size, this appears to be necessary to achieve good results
# * more good links for help especially the file preprocessing and the imagecropping portions early on (its difficult to learn this file processing part if you're not good in python already)
#
# https://www.kaggle.com/hooseygoose/directory-structure-and-moving-files
#
#
# http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html
#
#
# https://www.kaggle.com/sakarilukkarinen/demo-11-xception-and-dense-2
#
#
# https://www.kaggle.com/markupik/case2-retinopathy
#
#
# https://www.kaggle.com/sakarilukkarinen/demo-04-binary-classifier

# + [markdown] _uuid="67e8788d69706cf8b3337448715b4d8591d31f86"
# ## basic procedures to do 
# 1. sample the eyeimages's labels into dataframes randomly
# 2. upsize the images into 299x299 size
# 3. make imageDataGenerators for trainingset, validationset (and testsets possibly trainingset? its tedious to create those file directories though)
# 4. import Xception convNet and add some layers into it
# 5. train model (it will take long time)
# 6. plot accuracy and roc metrics, and also make tests with the testset (testset images would have to be cropped also, very tedious...)

# + _uuid="fc1ae35e1eb4b38e621f888ba4e4f2e2ca3259a6"
import pandas as pd
import numpy as np
import os
import keras
import matplotlib.pyplot as plt
from keras.layers import Dense,GlobalAveragePooling2D
from keras.applications import MobileNet
from keras.applications import VGG19
from keras.preprocessing import image
from keras.applications.mobilenet import preprocess_input
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Model
from keras.optimizers import Adam
from keras import Sequential
from keras.layers import Conv2D, Activation, MaxPooling2D, Dropout, Flatten, Dense
from keras import optimizers
import os, shutil, cv2, time



# + _uuid="40045d8faa8d1b1240e827f4639cfa335772219d"
"""Initial settings for training/validation, remember to choose!!!"""
EPOCHS = 30
BATCHSIZE = 16
TRAINSET_PERCENT = 0.75 ## percentage of trainingset from totalsamples (from [0, 1.0] )
SAMPLES = 4000  ## take huge sample of images, but only train&validate on subsets of allowed trainable/validatable images
TRAINSTEPS_PERCENT = 1.0  ## percentage of trainingset steps being done (from [0, 1.0] )
VALIDSTEPS_PERCENT = 1.0  ## percentage of validationset steps being done (from [0, 1.0] )
BALANCEDSET = False ## balance the df dataset 50:50 sick/healthy, otherwise simply random sample 
MODELNAME = "borrowedXceptionArchitecture" + ".h5"
trainConvLayers = True  ## train convLayers or freeze them
SEED = 1999-888
ROTANGLE = 10
ZOOM = 0.02
VERBOSE = 1  ## for modeltraining
IMAGESIZE = 299



source_dir = r"../input/preprocessed-diabetic-retinopathy-trainset/300_train/300_train/"
temp_dir = r"./temp/"
#test_dir = r"../input/300_test/300_test/"

# initialize multiple optimizers but you can only choose one at a time of course!
sgd = optimizers.SGD(lr=0.01/2, decay=1e-6, momentum=0.9, nesterov=True)
addyboi = optimizers.Adam(lr=0.01, decay=1e-5)
rmsprop = optimizers.RMSprop(lr=0.01/2)
basic_rms = optimizers.RMSprop()

"""remember to choose optimizer!!!"""
OPTIMIZER = basic_rms

# + _uuid="751696fea02258c282efdf3115630a1333010484"
# Convert categorical level to binary and randomly sample
import numpy as np
df = pd.read_csv(r"../input/preprocessed-diabetic-retinopathy-trainset/newTrainLabels.csv")
df['level'] = 1*(df['level'] > 0)

# balance classes
if BALANCEDSET:
    print('balancing dataset...')
    df = pd.concat([
        df[df['level']==0].sample(n=round(SAMPLES/2.0), random_state=SEED),
        df[df['level']==1].sample(n=round(SAMPLES/2.0), random_state=SEED)
    ]).sample(frac=1.0, random_state=SEED) # shuffle
else:
    print('raw dataset(unbalanced)...')
    df= df.sample(n=SAMPLES, random_state=SEED)


df.reset_index(drop=True, inplace=True) # I've had troubles with shuffled dataframes on some earlier Cognitive Mathematics labs,
                                        # and this seemed to prevent those
print(df.head())
print("")

# + _uuid="f0bad13666c487d6d7ca9927b3c2f5e20989fcdd"


# Create level histogram
df['level'].hist(bins = [0,1, 2], rwidth = 0.5, align = 'left');



# + _uuid="c98c598708fe7318439933f365d05dcd536b3bac"
# here we can split df into train/validation dataframes, and then we can use
# separated imagedatagenerators, such that we are able to use data augmentation 
# for only the traindata images
# the validdata images should be left raw as they are

setBoundary = round(SAMPLES * TRAINSET_PERCENT)
train_df = df[:setBoundary]
validate_df = df[setBoundary:]
train_df.reset_index(drop=True, inplace=True)
validate_df.reset_index(drop=True, inplace=True)

# + _uuid="ff6a11a37e7cc525ca67657c09ff5e4d03f83937"
# Here the issue with original demo4 code from Sakari was that there was the typeError from imageDataGenerators
# at least for myself, so that the error message suggested that I should convert level column to str
df["level"]= df.iloc[:,1].astype(str)
train_df["level"]= train_df.iloc[:,1].astype(str)
validate_df["level"]= validate_df.iloc[:,1].astype(str)

# + _uuid="c166458b7f9bd2869df89436ffbeac80a3f947a7"
#Test out images can be found
image_path = source_dir + df["image"][0] +".jpeg"
if  os.path.isfile(image_path)==False:
    raise Exception('Unable to find train image file listed on dataframe')
else:
    print('Train data frame and file path ready')
    

# + _uuid="10d2ca28f78a4253ee56b92072849e1c609af829"
# Prepare to crop images to larger size so that the network can work on them...
# apparently too small images are just shit for these convNeuralNetworks
# Create destination directory

try:
    os.mkdir(temp_dir)
    print('Created a directory:', temp_dir)
except:
    # Temp directory already exist, so clear it
    for file in os.listdir(temp_dir):  
        file_path = os.path.join(temp_dir, file)
        try:
            if os.path.isfile(file_path):
                os.unlink(file_path)
        except Exception as e:
            print(e)
    print(temp_dir, ' cleared.')

# + _uuid="f03ee3c201c1914a628b6aec9073163c8b6f142e"
# Crop the images to larger size from 100x100 upwards to atleast 300x300 
# some contestant winner had images 500x500
# I've tried this vgg19 with 100x100 and it was not good results

# Start timing
start = time.time()

# Crop and resize all images. Store them to dest_dir
print("Cropping and rescaling the images:")
for i, file in enumerate(df["image"]):
    try:
        fname = source_dir + file + ".jpeg"
        img = cv2.imread(fname)
    
        # Crop the image to the height
        h, w, c = img.shape
        if w > h:
            wc = int(w/2)
            w0 = wc - int(h/2)
            w1 = w0 + h
            img = img[:, w0:w1, :]
        # Rescale to N x N
        N = IMAGESIZE
        img = cv2.resize(img, (N, N))
        # Save
        new_fname = temp_dir  + file + ".jpeg"
        cv2.imwrite(new_fname, img)
    except:
        # Display the image name having troubles
        print("problemImagesFound:___ ", fname)
         
    # Print the progress for every N images
    if (i % 500 == 0) & (i > 0):
        print('{:} images resized in {:.2f} seconds.'.format(i, time.time()-start))

# End timing
print('Total elapsed time {:.2f} seconds.'.format(time.time()-start))
print('temp_dir was=' + temp_dir)

# + _uuid="2036f24122e914a612afb739000c77a18eb390be"
path0= temp_dir # specify directory where the f***ing files are
fileList = os.listdir(path0) # get the f***ing file list in the path directory
# list files
print(fileList[0])# print if you even found any f***ing files in this f***ing kaggle environment

# + _uuid="68b06c0baca1d0f98061d8b5f798b4a8f8adae26"
for i in range (5):
    print(df['image'][i])

# + _uuid="fc86eda59c6eb524133f27b28f19dc42d0b44091"
# We must convert the dataframes's image columns into .jpeg extension, so generators can find images into the dataframes
"""wtf is wrong with this code, why kaggle keeps raising warnings
why it is suddenly illegal to MODIFY the goddamn column to have ".jpeg" in the end for all values?!"""

print(df.head(5))
a=df.iloc[:,0] + ".jpeg"
b=train_df.iloc[:,0] + ".jpeg"
c=validate_df.iloc[:,0] + ".jpeg"
df['image']=a
train_df['image']=b
validate_df['image']=c
print(df.head(5))

# + _uuid="6dbd069823db8209e163d2ee811aaacf038163d8"
# Create image data generator
from keras.preprocessing.image import ImageDataGenerator
# I think that it would be possible also to use data augmentation in the training_generator only.
# Also the interview of kaggle contestants showed my own suspicions to be true that one should not use 
# image shear with eye data (the perfectly healthy human eyeball should be round, so you dont get eyeglasses)
# those contestant used rotation and mirrorings of the image as I recall, but small amounts of
#  zoom would not be too bad either, I reckon

# use data augmentation for training
traingen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=ROTANGLE,
    zoom_range=ZOOM,
    horizontal_flip=True,
    vertical_flip=True)

## just take the raw data for validation
validgen = ImageDataGenerator(
    rescale=1./255)

# Data flow for training
trainflow = traingen.flow_from_dataframe(
    dataframe = train_df, 
    directory = temp_dir,
    x_col = "image", 
    y_col = "level", 
    class_mode = "binary", # class_mode binary causes to infer classlabels automatically from y_col, at least according to keras documentation
    target_size = (IMAGESIZE, IMAGESIZE), 
    batch_size = BATCHSIZE,
    shuffle = True,
    seed = SEED)

# Data flow for validation
validflow = validgen.flow_from_dataframe(
    dataframe = validate_df, 
    directory = temp_dir,
    x_col = "image", 
    y_col = "level", 
    class_mode = "binary", 
    target_size = (IMAGESIZE, IMAGESIZE), 
    batch_size = BATCHSIZE,
    shuffle = False, # validgen doesnt need shuffle I think, according to teacher's readymade convnet example (?)
#    seed = SEED
)



# + _uuid="e7427e0dbff76b7f7e2378acbca3689f31afbd1e"
"""## We will try Juha Kopus model style for pre-trained VGG16. it was pretty bad in my opinoin, i tested it
x = base_model.output
x = Flatten(name='flatten')(x)
x = Dropout(0.3, seed=SEED)(x) # add dropout just in case
x = Dense(256,activation='relu')(x)
x = Dropout(0.3, seed=SEED)(x) # add dropout just in case
preds = Dense(1,activation='sigmoid')(x) #final layer with sigmoid activation"""
"""## Sakaris Xception model style for Xception convnet ##
## with added Dense model at the end of ConvNet"""
"""x = base_model.output ##this was some old code to make for MobileNet from example tutorial about convnets
x = GlobalAveragePooling2D()(x)
x = Flatten(name='flatten')(x)
x=Dropout(0.2, seed=SEED)(x)
x=Dense(1024,activation='relu')(x) #we add dense layers so that the model can learn more complex functions and classify for better results.
x=Dropout(0.3, seed=SEED)(x)
x=Dense(1024,activation='relu')(x) #dense layer 2
x=Dropout(0.3, seed=SEED)(x)
x=Dense(512,activation='relu')(x) #dense layer 3
x=Dropout(0.3, seed=SEED)(x)
preds=Dense(1,activation='sigmoid')(x) #final layer with sigmoid activation
"""



# + _uuid="cb25a07fd12f88d2e57b757e2d644ff061871d64"

from keras.applications.xception import Xception

base_model = Xception(include_top=False, input_shape=(IMAGESIZE, IMAGESIZE, 3)) # randomized initial weights, Xception_base

if not trainConvLayers:
    for layer in base_model.layers: # make convLayers un-trainable
        layer.trainable=False        
        
base_model.summary()

x = base_model.output

x = GlobalAveragePooling2D()(x)  
#x = Flatten(name='flatten')(x)  
x = Dropout(0.25, seed=SEED)(x) # add dropout just in case      
x = Dense(512, activation='relu')(x)
x = Dropout(0.4, seed=SEED)(x) # add dropout just in case      
x = Dense(512, activation='relu')(x)
x = Dropout(0.4, seed=SEED)(x) # add dropout just in case  
x = Dense(256,activation='relu')(x)
x = Dropout(0.4, seed=SEED)(x) # add dropout just in case
preds = Dense(1,activation='sigmoid')(x) #final layer with sigmoid activation

model=Model(inputs=base_model.input,outputs=preds)
#for i,layer in enumerate(model.layers):
#    print(i,layer.name)

print(model.summary())

model.compile(optimizer = OPTIMIZER,
             loss='binary_crossentropy', 
              metrics = ["accuracy"])

# + _uuid="56409c9656f3ae7b4df9ed2dc7fc0af170f7255c"

from keras.callbacks import ModelCheckpoint
from time import time, localtime, strftime
# Testing with localtime and strftime
print(localtime())
print(strftime('%Y-%m-%d-%H%M%S', localtime()))


# Calculate how many batches are needed to go through whole train and validation set
STEP_SIZE_TRAIN = round((trainflow.n // trainflow.batch_size) * 1.0 * TRAINSTEPS_PERCENT ) 
STEP_SIZE_VALID = round((validflow.n // validflow.batch_size) * 1.0 * VALIDSTEPS_PERCENT ) 

# Train and count time
model_name = strftime('Case2-%Y-%m-%d-%H%M%S', localtime()) + MODELNAME
print('modelname was=',model_name,'\n')

t1 = time()
h = model.fit_generator(generator = trainflow,
                    steps_per_epoch = STEP_SIZE_TRAIN,
                    validation_data = validflow,
                    validation_steps = STEP_SIZE_VALID,
                    epochs = EPOCHS,
                    verbose = VERBOSE)
t2 = time()
elapsed_time = (t2 - t1)

# Save the model
model.save(model_name)
print('')
print('Model saved to file:', model_name)
print('')

# Print the total elapsed time and average time per epoch in format (hh:mm:ss)
t_total = strftime('%H:%M:%S', localtime(t2 - t1))
t_per_e = strftime('%H:%M:%S', localtime((t2 - t1)/EPOCHS))
print('Total elapsed time for {:d} epochs: {:s}'.format(EPOCHS, t_total))
print('Average time per epoch:             {:s}'.format(t_per_e))



# + _uuid="107de288a1b98af3cd97602969ddbf1cb0a17dc5"
# get the currently trained model, and plot the accuracies and loss for training and validation

# %matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

epochs = np.arange(EPOCHS) + 1.0

f, (ax1, ax2) = plt.subplots(1, 2, figsize = (15,7))

def plotter(ax, epochs, h, variable):
    ax.plot(epochs, h.history[variable], label = variable)
    ax.plot(epochs, h.history['val_' + variable], label = 'val_'+variable)
    ax.set_xlabel('Epochs')
    ax.legend()

plotter(ax1, epochs, h, 'acc')
plotter(ax2, epochs, h, 'loss')
plt.show()




# + _uuid="8447c45a50bc74839445fc4f6af8ca4c4481d53a"
# get true values
y_true = validflow.classes
# note about predict_generator, 
# sometimes it happened that, if you put argument steps = STEP_SIZE_VALID, then
# that throws error because of mismatched steps amount somewhere, I think that the
# np.ceil(validgen.n/validgen.batch_size) seems to fix it
predict = model.predict_generator(validflow, steps= np.ceil(validflow.n / validflow.batch_size))
y_pred = 1*(predict > 0.5)

# + _uuid="753b868ed5333d265b58d4d29d14767671906a08"

# Calculate and print the metrics results
from sklearn.metrics import confusion_matrix, cohen_kappa_score, classification_report

cm = confusion_matrix(y_true, y_pred)
print('Lates Confusion matrix:')
print(cm)
print('')

cr = classification_report(y_true, y_pred)
print('Lates Classification report:')
print(cr)
print('')

from sklearn.metrics import accuracy_score
a = accuracy_score(y_true, (y_pred))
print(a)
print('Lates Accuracy with old decision point {:.4f} ==> {:.4f}'.format(0.5, a))
print('')


# + _uuid="301491f2584d66eba5dc9f77fe42eb47663369cd"

# Calculate and plot ROC-curve
# See: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html
from sklearn.metrics import roc_curve

fpr, tpr, thresholds = roc_curve(y_true, predict) 

plt.plot(fpr, tpr, color='darkorange', lw = 2)
plt.plot([0, 1], [0, 1], color='navy', lw = 2, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Lates Receiver operating characteristic curve')
plt.show()


# + _uuid="28c14b6aa91c6a5abc52b225047d6cec7c2df5f8"
from keras.models import load_model

sakarismodel = load_model("../input/sakarisbestmodelxception/best_demo11_SAKARISDEMO11.h5")
predictSakari = sakarismodel.predict_generator(validflow, steps= np.ceil(validflow.n / validflow.batch_size))
y_predSakari = 1*(predictSakari > 0.5)

print('Sakaris best model was as follows: \n')
# Calculate and print the metrics results
from sklearn.metrics import confusion_matrix, cohen_kappa_score, classification_report

cm = confusion_matrix(y_true, y_predSakari)
print('Sakaris Confusion matrix:')
print(cm)
print('')

cr = classification_report(y_true, y_predSakari)
print('Sakaris Classification report:')
print(cr)
print('')

from sklearn.metrics import accuracy_score
a = accuracy_score(y_true, (y_predSakari))
print(a)
print('Sakaris Accuracy with old decision point {:.4f} ==> {:.4f}'.format(0.5, a))
print('')


fpr0, tpr0, thresholds0 = roc_curve(y_true, y_predSakari) 

plt.plot(fpr0, tpr0, color='darkorange', lw = 2)
plt.plot([0, 1], [0, 1], color='navy', lw = 2, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Sakaris Receiver operating characteristic curve')
plt.show()


# + _uuid="843981a3627efaa0e59eeb781e79cbcd26843ac0"
# Clear the temporary directory
dest_dir = './temp/'
for file in os.listdir(dest_dir):
    file_path = os.path.join(dest_dir, file)
    try:
        if os.path.isfile(file_path):
            os.unlink(file_path)
    except Exception as e:
        print(e)
print(dest_dir, ' cleared.')
os.rmdir(dest_dir)
print(dest_dir,'Removed.')

# + [markdown] _uuid="d4583a6b2d3f1984432da2f710ea628b90d3a836"
# ## Conclusions
# In general, the benefits from this case2 to me was mostly about learning how to 
# work with convnets, but the downside was that the predictive results and model were quite bad.
#
# - I don't think it was possible to have good accuracies with small size images like 100x100 pre-processed dataset so that somehow the cropping and sizing-up the images seemed to make the crucial difference for the positive direction for model training/validation
# - imagenet weights were tried in another notebook version with the VGG19 model, but those imagenet weights didnt seem to work so well, at least with the 100x100 images
# - possibly it would have been best to utilize randomized weights for VGG19 and train the convLayers at least for even few epochs, and then attempt to freeze the convLayers and continue to train the dense layers to save some time
# - I tried to tweak the catsAndDogs model with dropouts, different learning rates, different optimizers and even increasing amounts of some convlayers and maxpools,
# but it appears that the basic architecture was not good enough for our purposes. It must have been underfitting which would indicate that architecture was too simplistic for the job, I think.
# - data augmentation was tried with zoom, rotation and image flippings, but the effect of data-augmentation did not stop model overfitting with basic Sakari's Xception architecture at least for myself.
# - I read from wikipedia that the human eye size has about max 2% difference in adult eye size, and the medical eye images should be pretty "fixed in place" in terms of the eye position because the patient is putting his head into a cradle to fully stabilize the head during an eye examination. But perhaps the blood veins inside the eye are "rotated" from person to person when comparing?
#
# - Xception convNet with dense, seemed to have good training and validation accuracies, but only 30 epochs were done for training. Perhaps that model could be better with larger trainingsets and more time spent on training the whole model. Maybe dropout could also be tweaked 
#
#
#

# + [markdown] _uuid="6eb6f9aae86b621a19a5c2a16cdffe3fddd9421e"
# ## Learning outcomes
#
# * most important lesson in this case2 for me was that when faced with image recognition, first you must try to overfit your model, then you can tweak it later.
# * second important lesson in this case2 was that if you want good results from actual testing, dont try to be a hero and re-invent the wheel, use ready convNet architectures at least first.
# * I learned how to work within kaggle environment, which was nice because it offloaded that computation to the cloud, so you could think about your next plan while the kernels are committing/running
# * later on, I read tutorial about how to use pre-trained convNet with keras and I managed to get some good results as soon as I cropped and upsized the images to about 299x299 from the old 100x100
# * I think I finally understood a lot of how that keras functionality works such as the imageDataGenerators and flow_from_dataframe.
# * gained more experience in working with the keras, because I had to read a lot into the documentation  (which parameters are there and what they do etc...)
# * I learned how the kaggle workflow works best at least for me if you want to try the same basic setup, but with different keras parameter such as switching optimizers, batchsizes, learning rates etc...
# * I also tested some basic style convnets such as the CIFAR-10 dataset model I introduced in the beginning, but the results were not good at least for me.
# * probably with hindsight the best strategy from the very beginning would have been to start immediately in the kaggle environment, then take a good pretrained convnet like inception or Vgg19 and just use that. In that way you would learn how to use the pre-trained convnet and also get good results. 

# + [markdown] _uuid="5d3d537e2dc8ba5b5b095432485dc300f0e5fe8e"
# ## my kaggle workflow
#
# 1. create basic notebook kernel which has the model and initial setups done, including the getting the data and training model
# 2. put epochs to 3 and run the whole notebook so it checks if it will crash (it should not take huge amount of time with only small amount of epochs)
# 3. now you can estimate how long it will take per epoch and write it on paper or something
# 4. put epochs back to whatever you want, then save notebook to your local pc (for safety purposes)
# 5. Then you could maybe fork that kernel again, then use that forked kernel to commit that one
# 6. then you should still be able to fork that original kernel how many times you want and make commits of those ones, with whateve changes you want to apply
# 7. then you can pretty much wait until the kernels are committed in the cloud and do something else or ponder your next move
# 8. as mentioned earlier you get your saved models in the committed kernels output section, if you want to download your trained models to your pc

# + [markdown] _uuid="b3902ecc27d3befd3473c3adf626081fb27b71e1"
#

# + [markdown] _uuid="2b7da276c37879040a4c3097797ce6b1f783853b"
#

StarcoderdataJupyterScriptsDedupFiltered