title: Neural network (perceptron) - visualizing decision boundary (as a hyperplane) when performing binary classification
body: <p>I would like to visualize the decision boundary for a simple neural network with only one neuron (3 inputs, binary output). I'm extracting the weights from a Keras NN model and then attempting to draw the surface plane using matplotlib. Unfortunately, the hyperplane is not appearing between the points on the scatter plot, but instead is displaying underneath all the data points (see output image).</p>
  
  <p>I am calculating the z-axis of the hyperplane using the equation
  <code>z = (d - ax - by) / c</code> for a hyperplane defined as <code>ax + by + cz = d</code></p>
  
  <p>Could somebody assist me with correctly constructing and displaying a hyperplane based on the NN weights?</p>
  
  <p><a href="https://i.stack.imgur.com/o35aK.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/o35aK.jpg" alt="Output:"></a></p>
  
  <p>The goal here is to classify individuals into two groups (diabetes or no diabetes), based on 3 predictor variables using a public dataset (<a href="https://www.kaggle.com/uciml/pima-indians-diabetes-database" rel="nofollow noreferrer">https://www.kaggle.com/uciml/pima-indians-diabetes-database</a>).</p>
  
  <pre><code>%matplotlib notebook
  
  import pandas as pd
  import numpy as np
  from keras import models
  from keras import layers
  import matplotlib.pyplot as plt
  from mpl_toolkits import mplot3d
  
  EPOCHS = 2
  
  #Data source: https://www.kaggle.com/uciml/pima-indians-diabetes-database
  ds = pd.read_csv('diabetes.csv', sep=',', header=0)
  
  #subset and split
  X = ds[['BMI', 'DiabetesPedigreeFunction', 'Glucose']]
  Y = ds[['Outcome']]
  
  #construct perceptron with 3 inputs and a single output
  model = models.Sequential()
  layer1 = layers.Dense(1, activation='sigmoid', input_shape=(3,))
  model.add(layer1)
  
  model.compile(optimizer='adam',
                loss='binary_crossentropy',
                metrics=['accuracy'])
  
  #train perceptron
  history = model.fit(x=X, y=Y, epochs=EPOCHS)
  
  #display accuracy and loss
  epochs = range(len(history.epoch))
  
  plt.figure()
  plt.plot(epochs, history.history['accuracy'])
  plt.xlabel('Epochs')
  plt.ylabel('Accuracy')
  
  plt.figure()
  plt.plot(epochs, history.history['loss'])
  plt.xlabel('Epochs')
  plt.ylabel('Loss')
  
  plt.show()
  
  #extract weights and bias from model
  weights = model.layers[0].get_weights()[0]
  biases = model.layers[0].get_weights()[1]
  
  w1 = weights[0][0] #a
  w2 = weights[1][0] #b
  w3 = weights[2][0] #c
  b = biases[0]      #d
  
  #construct hyperplane: ax + by + cz = d
  a,b,c,d = w1,w2,w3,b
  
  x_min = ds.BMI.min()
  x_max = ds.BMI.max()
  
  x = np.linspace(x_min, x_max, 100)
  
  y_min = ds.DiabetesPedigreeFunction.min()
  y_max = ds.DiabetesPedigreeFunction.max()
  
  y = np.linspace(y_min, y_max, 100)
  
  Xs,Ys = np.meshgrid(x,y)
  Zs = (d - a*Xs - b*Ys) / c
  
  #visualize 3d scatterplot with hyperplane
  fig = plt.figure(num=None, figsize=(9, 9), dpi=100, facecolor='w', edgecolor='k')
  ax = fig.gca(projection='3d')
  
  ax.plot_surface(Xs, Ys, Zs, alpha=0.45)
  
  ax.scatter(ds.BMI, ds.DiabetesPedigreeFunction, ds.Glucose, c=ds.Outcome)
  
  ax.set_xlabel('BMI')
  ax.set_ylabel('DiabetesPedigreeFunction')
  ax.set_zlabel('Glucose')
  </code></pre>