How does #value_callback really work (nested elements, ...)? EDIT: Separated 4th question. 

I've just recently started working with custom elements and already came across some weird behavior / questions I'd like to see answered.

1) How does #value_callback work, what should you return in it, etc. ? 

Imagine you have pretty simple custom element without set #element_validate:

function TST_element_process($element, &amp;$form_statem, $form) {
  $element['testing_test'] = array(
    '#type' =&gt; 'textfield',
    '#default_value' =&gt; 'ABC',
  );

  return $element;
}
function TST_value_callback_1($element, $input = FALSE, &amp;$form_state) {
  //example of input retrieving magic
  return ($input['testing_test'] . '_valc_');
}
function TST_value_callback_2($element, $input = FALSE, &amp;$form_state) {
  //example of input retrieving magic
  return array($input['testing_test'] . '_valc_');
}

'TST' =&gt; array(
  '#input' =&gt; TRUE,
  '#tree' =&gt; TRUE,
  '#process' =&gt; array('TST_element_process'),
  //'#element_validate' =&gt; array('TST_element_validate'),
  '#value_callback' =&gt; 'TST_value_callback_1', // or TST_value_callback_2
),


And form:

$ar['inp'] = array('#type' =&gt; 'TST');


1A) When you use TST_value_callback_1, the callback gets called and runs but doesn't alter the result value in any way (in submit the $form_state['values']['inp']['testing_test'] contains pure input -> ABC). 

1B) In case of TST_value_callback_2 the $form_state['values']['inp']['testing_test'] still contains unaltered input but there's also value in $form_state['values']['inp'][0] which contains the 'right' value (ABC_valc_).

My questions are:

1.1 what exactly are you supposed to return in #value_callback and why do the individual callbacks work in the way they do (or don't do in case of the first one). 

1.2 is there any way how to make the #value_callback store the data in 'element root' ($form_state['values']['inp']), i.e. in the same place as if I used form_set_value in #element_validate?



2) How do individual #value_callback work on custom element that is made of other elements?

Let's say you have a custom element (A) that is made of a nested set of other elements with set #value_callback. 

for($a = 0; $a &lt; 10; $a++) {
  $element['indiv_elems'][] = array(
    '#type' =&gt; 'indiv_element',
  );
}


Then a number of things can appear in $form_state[values][&lt;element_name_in_FAPI_array&gt;] depending on how you approach #value_callback in the custom event (A) (and some of those things make little sense to me).

2A) When you don't use #value_callback in (A) at all, the $form_state[values][&lt;element_name_in_FAPI_array&gt;] will contain array of raw inputs (not processed by its #value_callback function) from each phone_numbers element.

2B) When you set #value_callback in (A) but let it return (let's say) empty array(), the $form_state[values][&lt;element_name_in_FAPI_array&gt;] will contain array of  inputs from each indiv_element element processed by their (individual  #value_callback function) . 

My question is:

2.1) Why do the values not get processed by individual #value_callback when parent element's #value_callback is not set. Basically why is there difference between 2A) and 2B).



3) Do I understand this flowchart  correctly that #value_callback is called before #element_validate?



I'm also aware that it will be quite time consuming to answer this question, so many thanks and love ahead! :)
One thing i want to add before replying to your questions. Drupal core FAPI doesn't allow multiple #value_callback. It allow only one #value_callback per element type. Even You don't have to add #value_callback key in your hook_element_info() if your value callback follow the format form_type_' . $element['#type'] . '_value'.
// See code in include/form.inc from function _form_builder_handle_input_element line 2034

$value_callback = !empty($element['#value_callback']) ? $element['#value_callback'] : 'form_type_' . $element['#type'] . '_value';


            

1) How does #value_callback work, what should you return in it, etc. ?

value_callback provides a way to process values before they are returned to the form submit function. You need to provide a #value_callback if you want to do fancy processing on the input. If you just want #value set to #default_value, you don't need this hook at all, since that will happen anyway. This function is mainly useful if you need to do something non-standard with the value, like transpose an array into an option list.
Value callbacks should consider three conditions:


Input is being provided directly. This happens when a form is submitted.
No input is provided, but the field definition has a default value.
No input is provided and there is no default value.

1.1) what exactly are you supposed to return in #value_callback and why do the individual callbacks work in the way they do (or don't do in case of the first one).
So value callback is supposed to check the $input !== FALSE and then modify the $input value as per your logic. If its false that means $input value is not provided in this case you can return the $element['#defalut_value']. So in your first value callback TST_value_callback_1 if you want to make that work you have to return the value with the same array key something like.
function TST_value_callback_1($element, $input = FALSE, &amp;$form_state) {
  //example of input retrieving magic
  $input['testing_test'] = $input['testing_test'] . '_valc_';
  return $input;
}

In your second value callback TST_value_callback_2  it was adding value to $form_state['values']['inp'][0] because you was returning an array with index 0 having the value ABC_valc_ like array(0 =&gt; 'ABC_valc_');
// code from include/form.inc from function _form_builder_handle_input_element line 2109

// Set the element's value in $form_state['values'], but only, if its key
// does not exist yet (a #value_callback may have already populated it).
if (!drupal_array_nested_key_exists($form_state['values'], $element['#parents'])) {
  form_set_value($element, $element['#value'], $form_state);
}

1.2 is there any way how to make the #value_callback store the data in 'element root' ($form_state['values']['inp']), i.e. in the same place as if I used form_set_value in #element_validate?
Setting the #tree to false in your form element should do the trick.
$ar['inp'] = array(
  '#type' =&gt; 'TST',
  '#tree' =&gt; FALSE,
  );

you will be able to get the value in $form_state['values']['inp'] but in this case #value_callback will not do any affect, and $input in #value_callback is set to NULL  i don't know why. So you better use the ($form_state['values']['inp']['testing_test'])
2.1) Is there any way to enable 2.2 behavior without setting #value_callback in ?
2.2) Why are individual #value_callback functions run (and their outputs are saved to $form_state) only when parent (A) #value_callback is run (even if it doesn't reference or call the individual functions in any way)?
Hopefully my above explanation help you to understand the role of #value_callback. Even though custom element that is made of a nested set of other elements, are also elements so individual value callback will be called for each subelement. There is too much too explain Please take a look at _form_builder_handle_input_element
3) Do I understand this flowchart correctly that #value_callback is called before #element_validate?
yes #value_callback is called before the #element_validate callback. For more explanation See Creating Custom Elements
For practical example see examples module.
See also:
https://api.drupal.org/api/examples/form_example%21form_example_elements.inc/function/form_type_form_example_checkbox_value/7
https://api.drupal.org/api/examples/form_example%21form_example_elements.inc/function/form_example_phonenumber_combined_value/7