How to efficiently (without looping) get data from tensor predicted by a torchscript in C++?
I am calling a torchscript (neural network serialized from Python) from a C++ program:
  // define inputs
  int batch = 3; // batch size
  int n_inp = 2; // number of inputs
  double I[batch][n_inp] = {{1.0, 1.0}, {2.0, 3.0}, {4.0, 5.0}}; // some random input
  std::cout &lt;&lt; &quot;inputs&quot; &quot;\n&quot;;  // print inputs
  for (int i = 0; i &lt; batch; ++i)
  {    
    std::cout &lt;&lt; &quot;\n&quot;;
    for (int j = 0; j &lt; n_inp; ++j)
    {
      std::cout &lt;&lt; I[i][j] &lt;&lt; &quot;\n&quot;;
    }
  }
  
  // prepare inputs for feeding to neural network
  std::vector&lt;torch::jit::IValue&gt; inputs;
  inputs.push_back(torch::from_blob(I, {batch, n_inp}, at::kDouble));

  // deserialize and load scriptmodule
  torch::jit::script::Module module;
  module = torch::jit::load(&quot;Net-0.pt&quot;);

  // do forward pass
  auto outputs = module.forward(inputs).toTensor();


Usually, to get data from the outputs, the following (element-wise) operation is performed:
  // get data from outputs
  std::cout &lt;&lt; &quot;outputs&quot; &lt;&lt; &quot;\n&quot;;
  int n_out = 1;
  double outputs_data[batch][n_out];
  for (int i = 0; i &lt; batch; i++) 
  {
    for (int j = 0; j &lt; n_out; j++)
    {
      outputs_data[i][j] = outputs[i][j].item&lt;double&gt;();
      std::cout &lt;&lt; outputs_data[i][j] &lt;&lt; &quot;\n&quot;;
    }
  }


However, such looping using .item is highly inefficient (in the actual code I will have millions of points predicted at each time step). I want to get data from outputs directly (without looping over elements). I tried:
  int n_out = 1;
  double outputs_data[batch][n_out];
  outputs_data = outputs.data_ptr&lt;double&gt;();

However, it is giving the error:
error: incompatible types in assignment of ‘double*’ to ‘double [batch][n_out]’
   outputs_data = outputs.data_ptr&lt;double&gt;();
                                           ^

Note, that type of outputs_data is fixed to double and cannot be changed.

It is necessary to make a deep copy as follows:
double outputs_data[batch];
std::memcpy(outputs_data, outputs.data_ptr&lt;dfloat&gt;(), sizeof(double)*batch);

