How do i check the duplicate entry in the table?
One way of doing that is using the hook_form_alter and add the custom validation function, and in that custom validation you can check if the enterd email already exists in the database than set error, somewhat like below:

/**
 * Implements hook_form_alter.
 */
function MYMODULE_form_alter(&amp;$form, &amp;$form_state, $form_id) {
  // Check the form id of your form.
  if ($form_id === 'YOUR_FORM_ID_WHICH_YOU_WANT_TO_EDIT') {
    $form['#validate'][] = 'check_email_address_exists';
  }
}

function check_email_address_exists(&amp;$form, &amp;$form_state) {
  // Get the value of email field. if you are under than use: "dpm($form_state['values'])"
  $email_entered = $form_state['values']['EMAIL_FIELD_NAME'];

  // Generate the query somewhat like below, For Example I am using users table's "uid".
  $query = db_select('users', 'u');
  $query-&gt;fields('u', array('uid'));
  $query-&gt;condition('u.mail', $email_entered);

  // This will return the count of the fields found.
  $row_count = $query-&gt;countQuery()-&gt;execute()-&gt;fetchField(); 

  if (isset($row_count) &amp;&amp; ($row_count &gt; 0)) {
    form_set_error('EMAIL_ADDRESS_FIELD_NAME', t('This email already exists, please try with another email  address!'));
  }
}


In order it to work you have to change the values as per you requirement, and don't forget to modify the db_select query as per your requirement, and last CLEAR THE CACHE after implementing the same.