Drupal Commerce: How to validate payment method settings?
I found a solution - because it is a FORM I using the FORMAPI. More accurate hook_form_alter() to add my custom validation handler and next is trivial. Code:

/**
 * Implements hook_form_alter()
 */
function commerce_mymodule_form_alter(&amp;$form, &amp;$form_state, $form_id) {
  if ($form_id == 'rules_ui_edit_element') {
    if ($form_state['element_settings']['payment_method']['method_id'] == 'commerce_mypaymentmethod') {
      $form['#validate'][] = 'commerce_mypaymentmethod_settings_validate';
    }
  }
}


and validation function

function commerce_mymodule_settings_validate(&amp;$form, &amp;$form_state) {  
  $deadline = (int) $form_state['values']['parameter']['payment_method']['settings']['payment_method']['settings']['mymodule_deadline'];

  if ($deadline &lt; 1 || $deadline &gt; 30) {
    form_set_error('', t('Enter only digit positive value between 1 -30 (days) in Order payment deadline. @value', array('@value' =&gt; $deadline)));
  }
}