Deduct 10% transaction fee when user donates userpoints to another user using rules - Drupal 7
If you see userpoints_donate.module userpoints_donation_giveform_submit function is reponsible to award points to another user

/**
 * Submit function to give another user points.
 */
function userpoints_donation_giveform_submit($form, &amp;$form_state) {
  global $user;

  $points = $form_state['values']['amount'];
  $to = $form_state['validated_account'];

  $tid = NULL;
  if (isset($form_state['values']['tid'])) {
    $tid = $form_state['values']['tid'];
  }

  $params = array(
    'uid'    =&gt; $to-&gt;uid,
    'points' =&gt; $points,
    'operation'  =&gt; 'userpoints_donation_received',
    'entity_type' =&gt; 'user',
    'entity_id' =&gt; $user-&gt;uid,
    'description' =&gt; $form_state['values']['reason'],
    'tid'    =&gt; $tid,
  );
  userpoints_userpointsapi($params);

  $params = array(
    'uid'    =&gt; $user-&gt;uid,
    'points' =&gt; -$points,
    'operation'  =&gt; 'userpoints_donation_given',
    'entity_type' =&gt; 'user',
    'entity_id' =&gt; $to-&gt;uid,
    'description' =&gt; $form_state['values']['reason'],
    'tid'    =&gt; $tid,
  );
  userpoints_userpointsapi($params);
}


As you see above 

userpoints_userpointsapi($params); is called to award $points with transaction operation userpoints_donation_received .

You can use hook_userpoints($op, $params = array()) to act upon certain operations. When other modules award  points to a user, your hook will be called, among others.


  $op: The operation to be acted upon.
      'setting'
        Pass a field set and fields that would be displayed in the userpoints
        settings page. For example, this can be used for your program to ask
        the admin to set a number of points for certain actions your module
        performs. The function should return an array object conforming to
        FormsAPI structure.
  
  'points before'   Calls your module, and others, before the points are
  processed. You can   prevent points from being awarded by returning
  FALSE.
  
  'points after'   Calls your module, and others, after points are
  processed. You can take   certain actions if you so wish. Return value
  is ignored.
  
  The $params variable is the original $params array as sent to
  userpoints_userpointsapi



Using "Points after" $op and  in $params check for operation "userpoints_donation_received"  you make a call to deduct 10 % points using..


userpoints_userpointsapi($params); API function..

&lt;?php
function YOURMODULENAME_userpoints($op, $params) {
  if ($op == "points after") {
    if ($params['operation'] == "userpoints_donation_received") {
      $transaction_params = $params;
      $transaction_params['points'] = -( $transaction_params['points'] / 10);
      $transaction_params['operation'] = "userpoints_donation_transaction_fee";
      $transaction_params['description'] = "Transaction Fee";
      userpoints_userpointsapi($transaction_params);
    }
  }
}