src/models/train_model.py
def build_embedding_network_3(X_train, X_val, cat_cols, cols_to_keep, n_num=120, n1=150, n2=50, d=False, lr=0.001, verbose=False):
    """Builds a neural network model with entity embedding for categorical variables.

    The function builds the neural network, for which it creates entity embedding for each categorical feature specified
    in cat_cols argument. Numerical features are projected ot a dense layer. Entity embeddings are concatenated with
    layer numerical features are projected and project to a dense layer with n1 neurons. There are additional two layers
    with n2 neurons each before projecting to the output layer.

    :param X_train: training set before preprocessing.
    :param X_val: validation set before preprocessing. The training and validation sets are need to calculate the
    dimensions of the entity embeddings.
    :param cat_cols: list with the names of categorical columns.
    :param cols_to_keep: list with the names of the columns that are selected for the model.
    :param n_num: number of neurons in the layer receiving input from the numerical feature matrix.
    :param n1: number of neurons in the layer that takes input from concatenated layer of entity embeddings and
    feature.
    :param n2: number of neurons in 2 dense layers prior to the output layer.
    :param d: dropout setting for the hidden layers before the output layer.
    :param lr: learning rate of the model
    :param verbose: if True prints the output with the size of the models.
    :return: neural network model.
    """
    inputs = []
    embeddings = []
    for categorical_var in cat_cols:
        if verbose:
            print("------------------------------------------------------------------")
            print("for categorical column ", categorical_var)
        input_cat = Input(shape=(1,))
        el_index = cols_to_keep.index(categorical_var)
        no_of_unique_cat = np.unique(np.concatenate((X_train, X_val), axis=0)[:, el_index]).size
        if verbose:
            print("number of unique cat", no_of_unique_cat)
        embedding_size = min(np.ceil((no_of_unique_cat) / 2), 50)
        embedding_size = int(embedding_size)
        if verbose:
            print("embedding_size set as ", embedding_size)
        embedding = Embedding(no_of_unique_cat + 1, embedding_size, input_length=1)(input_cat)
        embedding = Reshape(target_shape=(embedding_size,))(embedding)
        inputs.append(input_cat)
        embeddings.append(embedding)

    other_cols = [not c in cat_cols for c in cols_to_keep]
    len_other_cols = len([c for c in other_cols if c])
    if verbose:
        print("------------------------------------------------------------------")
        print('Numeric columns')
        print('Number of columns', len_other_cols)
    input_numeric = Input(shape=(len_other_cols,))
    embedding_numeric = Dense(n_num)(input_numeric)
    inputs.append(input_numeric)
    embeddings.append(embedding_numeric)

    x = Concatenate()(embeddings)
    x = Dense(n1, activation='relu')(x)
    if d:
        x = Dropout(d)(x)
    x = Dense(n1, activation='relu')(x)
    if d:
        x = Dropout(d)(x)
    x = Dense(n2, activation='relu')(x)
    if d:
        x = Dropout(d)(x)
    output = Dense(1, activation='sigmoid')(x)

    model = Model(inputs, output)

    optimizer = keras.optimizers.Adam(lr=lr)

    model.compile(loss='binary_crossentropy', optimizer=optimizer)
    return model
Builds a neural network model with entity embedding for categorical variables.

    The function builds the neural network, for which it creates entity embedding for each categorical feature specified
    in cat_cols argument. Numerical features are projected ot a dense layer. Entity embeddings are concatenated with
    layer numerical features are projected and project to a dense layer with n1 neurons. There are additional two layers
    with n2 neurons each before projecting to the output layer.

    :param X_train: training set before preprocessing.
    :param X_val: validation set before preprocessing. The training and validation sets are need to calculate the
    dimensions of the entity embeddings.
    :param cat_cols: list with the names of categorical columns.
    :param cols_to_keep: list with the names of the columns that are selected for the model.
    :param n_num: number of neurons in the layer receiving input from the numerical feature matrix.
    :param n1: number of neurons in the layer that takes input from concatenated layer of entity embeddings and
    feature.
    :param n2: number of neurons in 2 dense layers prior to the output layer.
    :param d: dropout setting for the hidden layers before the output layer.
    :param lr: learning rate of the model
    :param verbose: if True prints the output with the size of the models.
    :return: neural network model.
    
Builds a neural network model with entity embedding for categorical variables.
The function builds the neural network, for which it creates entity embedding for each categorical feature specified
in cat_cols argument. Numerical features are projected ot a dense layer. Entity embeddings are concatenated with
layer numerical features are projected and project to a dense layer with n1 neurons. There are additional two layers
with n2 neurons each before projecting to the output layer.
Builds a neural network model with entity embedding for categorical variables . The function builds the neural network for which it creates entity embedding for each categorical feature specified in cat_cols argument . Numerical features are projected ot a dense layer . Entity embeddings are concatenated with layer numerical features are projected and project to a dense layer with n1 neurons . There are additional two layers with n2 neurons each before projecting to the output layer .
def build_embedding_network_3(X_train, X_val, cat_cols, cols_to_keep, n_num=120, n1=150, n2=50, d=False, lr=0.001, verbose=False):
    inputs = []
    embeddings = []
    for categorical_var in cat_cols:
        if verbose:
            print("------------------------------------------------------------------")
            print("for categorical column ", categorical_var)
        input_cat = Input(shape=(1,))
        el_index = cols_to_keep.index(categorical_var)
        no_of_unique_cat = np.unique(np.concatenate((X_train, X_val), axis=0)[:, el_index]).size
        if verbose:
            print("number of unique cat", no_of_unique_cat)
        embedding_size = min(np.ceil((no_of_unique_cat) / 2), 50)
        embedding_size = int(embedding_size)
        if verbose:
            print("embedding_size set as ", embedding_size)
        embedding = Embedding(no_of_unique_cat + 1, embedding_size, input_length=1)(input_cat)
        embedding = Reshape(target_shape=(embedding_size,))(embedding)
        inputs.append(input_cat)
        embeddings.append(embedding)
    other_cols = [not c in cat_cols for c in cols_to_keep]
    len_other_cols = len([c for c in other_cols if c])
    if verbose:
        print("------------------------------------------------------------------")
        print('Numeric columns')
        print('Number of columns', len_other_cols)
    input_numeric = Input(shape=(len_other_cols,))
    embedding_numeric = Dense(n_num)(input_numeric)
    inputs.append(input_numeric)
    embeddings.append(embedding_numeric)
    x = Concatenate()(embeddings)
    x = Dense(n1, activation='relu')(x)
    if d:
        x = Dropout(d)(x)
    x = Dense(n1, activation='relu')(x)
    if d:
        x = Dropout(d)(x)
    x = Dense(n2, activation='relu')(x)
    if d:
        x = Dropout(d)(x)
    output = Dense(1, activation='sigmoid')(x)
    model = Model(inputs, output)
    optimizer = keras.optimizers.Adam(lr=lr)
    model.compile(loss='binary_crossentropy', optimizer=optimizer)
    return model
def build_embedding_network_3 ( X_train , X_val , cat_cols , cols_to_keep , n_num = 120 , n1 = 150 , n2 = 50 , d = False , lr = 0.001 , verbose = False ) : inputs = [ ] embeddings = [ ] for categorical_var in cat_cols : if verbose : print ( "------------------------------------------------------------------" ) print ( "for categorical column " , categorical_var ) input_cat = Input ( shape = ( 1 , ) ) el_index = cols_to_keep . index ( categorical_var ) no_of_unique_cat = np . unique ( np . concatenate ( ( X_train , X_val ) , axis = 0 ) [ : , el_index ] ) . size if verbose : print ( "number of unique cat" , no_of_unique_cat ) embedding_size = min ( np . ceil ( ( no_of_unique_cat ) / 2 ) , 50 ) embedding_size = int ( embedding_size ) if verbose : print ( "embedding_size set as " , embedding_size ) embedding = Embedding ( no_of_unique_cat + 1 , embedding_size , input_length = 1 ) ( input_cat ) embedding = Reshape ( target_shape = ( embedding_size , ) ) ( embedding ) inputs . append ( input_cat ) embeddings . append ( embedding ) other_cols = [ not c in cat_cols for c in cols_to_keep ] len_other_cols = len ( [ c for c in other_cols if c ] ) if verbose : print ( "------------------------------------------------------------------" ) print ( 'Numeric columns' ) print ( 'Number of columns' , len_other_cols ) input_numeric = Input ( shape = ( len_other_cols , ) ) embedding_numeric = Dense ( n_num ) ( input_numeric ) inputs . append ( input_numeric ) embeddings . append ( embedding_numeric ) x = Concatenate ( ) ( embeddings ) x = Dense ( n1 , activation = 'relu' ) ( x ) if d : x = Dropout ( d ) ( x ) x = Dense ( n1 , activation = 'relu' ) ( x ) if d : x = Dropout ( d ) ( x ) x = Dense ( n2 , activation = 'relu' ) ( x ) if d : x = Dropout ( d ) ( x ) output = Dense ( 1 , activation = 'sigmoid' ) ( x ) model = Model ( inputs , output ) optimizer = keras . optimizers . Adam ( lr = lr ) model . compile ( loss = 'binary_crossentropy' , optimizer = optimizer ) return model
Builds a neural network model with entity embedding for categorical variables.
Builds a neural network model with entity embedding for categorical variables .
"""Builds a neural network model with entity embedding for categorical variables.

    The function builds the neural network, for which it creates entity embedding for each categorical feature specified
    in cat_cols argument. Numerical features are projected ot a dense layer. Entity embeddings are concatenated with
    layer numerical features are projected and project to a dense layer with n1 neurons. There are additional two layers
    with n2 neurons each before projecting to the output layer.

    :param X_train: training set before preprocessing.
    :param X_val: validation set before preprocessing. The training and validation sets are need to calculate the
    dimensions of the entity embeddings.
    :param cat_cols: list with the names of categorical columns.
    :param cols_to_keep: list with the names of the columns that are selected for the model.
    :param n_num: number of neurons in the layer receiving input from the numerical feature matrix.
    :param n1: number of neurons in the layer that takes input from concatenated layer of entity embeddings and
    feature.
    :param n2: number of neurons in 2 dense layers prior to the output layer.
    :param d: dropout setting for the hidden layers before the output layer.
    :param lr: learning rate of the model
    :param verbose: if True prints the output with the size of the models.
    :return: neural network model.
    """
neural network model.
neural network model .
X_train
training set before preprocessing.
training set before preprocessing .
X_val
validation set before preprocessing. The training and validation sets are need to calculate the
dimensions of the entity embeddings.
validation set before preprocessing . The training and validation sets are need to calculate the dimensions of the entity embeddings .
cat_cols
list with the names of categorical columns.
list with the names of categorical columns .
cols_to_keep
list with the names of the columns that are selected for the model.
list with the names of the columns that are selected for the model .
n_num
number of neurons in the layer receiving input from the numerical feature matrix.
number of neurons in the layer receiving input from the numerical feature matrix .
n1
number of neurons in the layer that takes input from concatenated layer of entity embeddings and
feature.
number of neurons in the layer that takes input from concatenated layer of entity embeddings and feature .
n2
number of neurons in 2 dense layers prior to the output layer.
number of neurons in 2 dense layers prior to the output layer .
d
dropout setting for the hidden layers before the output layer.
dropout setting for the hidden layers before the output layer .
lr
learning rate of the model
learning rate of the model
verbose
if True prints the output with the size of the models.
if True prints the output with the size of the models .