class_docstr: A set of predefined models to approximate Q or log of pi (policy).
  
  The loss function needs to be 'cross_entropy_with_softmax' for policy
  gradient methods.
docstr: Dueling network to approximate Q function.
  
  See paper at https://arxiv.org/pdf/1511.06581.pdf.
  
  Args:
      shape_of_inputs: tuple of array (input) dimensions.
      number_of_outputs: dimension of output, equals the number of
          possible actions.
      model_hidden_layers: in the form of "[comma-separated integers,
          [comma-separated integers], [comma-separated integers]]". Each
          integer is the number of nodes in a hidden layer.The
          first set of integers represent the shared component in dueling
          network. The second set correponds to the state value function
          V and the third set correponds to the advantage function A.
      loss_function: if not specified, use squared loss by default.
      use_placeholder_for_input: if true, inputs have to be replaced
          later with actual input_variable.
  
  Returns: a Python dictionary with string-valued keys including
      'inputs', 'outputs', 'loss' and 'f'.
docstr_tok: ["Dueling","network","to","approximate","Q","function",".","See","paper","at","https",":","\/\/arxiv.org\/pdf\/1511.06581.pdf",".","Args",":","shape_of_inputs",":","tuple","of","array","(","input",")","dimensions",".","number_of_outputs",":","dimension","of","output",",","equals","the","number","of","possible","actions",".","model_hidden_layers",":","in","the","form","of","``","[","comma-separated","integers",",","[","comma-separated","integers","]",",","[","comma-separated","integers","]","]","''",".","Each","integer","is","the","number","of","nodes","in","a","hidden","layer.The","first","set","of","integers","represent","the","shared","component","in","dueling","network",".","The","second","set","correponds","to","the","state","value","function","V","and","the","third","set","correponds","to","the","advantage","function","A.","loss_function",":","if","not","specified",",","use","squared","loss","by","default",".","use_placeholder_for_input",":","if","true",",","inputs","have","to","be","replaced","later","with","actual","input_variable",".","Returns",":","a","Python","dictionary","with","string-valued","keys","including","'inputs","'",",","'outputs","'",",","'loss","'","and","'","f","'","."]
code: def dueling_network(
      shape_of_inputs,
      number_of_outputs,
      model_hidden_layers,
      loss_function=None,
      use_placeholder_for_input=False,
  ):
      """Dueling network to approximate Q function.
  
      See paper at https://arxiv.org/pdf/1511.06581.pdf.
  
      Args:
          shape_of_inputs: tuple of array (input) dimensions.
          number_of_outputs: dimension of output, equals the number of
              possible actions.
          model_hidden_layers: in the form of "[comma-separated integers,
              [comma-separated integers], [comma-separated integers]]". Each
              integer is the number of nodes in a hidden layer.The
              first set of integers represent the shared component in dueling
              network. The second set correponds to the state value function
              V and the third set correponds to the advantage function A.
          loss_function: if not specified, use squared loss by default.
          use_placeholder_for_input: if true, inputs have to be replaced
              later with actual input_variable.
  
      Returns: a Python dictionary with string-valued keys including
          'inputs', 'outputs', 'loss' and 'f'.
      """
      # input/output
      inputs = (
          C.ops.placeholder(shape=shape_of_inputs)
          if use_placeholder_for_input
          else C.ops.input_variable(
              shape=shape_of_inputs, dtype=np.float32
          )
      )
      outputs = C.ops.input_variable(
          shape=(number_of_outputs,), dtype=np.float32
      )
  
      # network structure
      (
          shared_hidden_layers,
          v_hidden_layers,
          a_hidden_layers,
      ) = Models._parse_dueling_network_structure(model_hidden_layers)
      # shared layers
      s = C.layers.For(
          range(len(shared_hidden_layers)),
          lambda h: C.layers.Dense(
              shared_hidden_layers[h], activation=C.ops.relu
          ),
      )(inputs)
      # Value function
      v = C.layers.Sequential(
          [
              C.layers.For(
                  range(len(v_hidden_layers)),
                  lambda h: C.layers.Dense(
                      v_hidden_layers[h], activation=C.ops.relu
                  ),
              ),
              C.layers.Dense(1, activation=None),
          ]
      )(s)
      # Advantage function
      a = C.layers.Sequential(
          [
              C.layers.For(
                  range(len(a_hidden_layers)),
                  lambda h: C.layers.Dense(
                      a_hidden_layers[h], activation=C.ops.relu
                  ),
              ),
              C.layers.Dense(number_of_outputs, activation=None),
          ]
      )(s)
      # Q = V + A - avg(A)
      avg_a = C.layers.AveragePooling((number_of_outputs,))(a)
      q = v + a - avg_a
  
      if loss_function is None:
          loss = C.losses.squared_error(q, outputs)
      else:
          loss = loss_function(q, outputs)
  
      return {
          "inputs": inputs,
          "outputs": outputs,
          "f": q,
          "loss": loss,
      }
  
file_path: CNTK/bindings/python/cntk/contrib/deeprl/agent/shared/models.py
imports: import cntk
  import numpy
  import ast
label_desc: Use image node_id 2 for calling the Models obj's underlying member method code with example usage: obj.dueling_network(shape_of_inputs, number_of_outputs, model_hidden_layers, loss_function, use_placeholder_for_input) and returns: dict