dotCall64: An Efficient Interface to Compiled C/C++ and Fortran Code Supporting Long Vectors
Florian Gerber Kaspar Mösinger Reinhard Furrer



Contributed research article
XX
YY
20ZZ
AAAA












dotCall64: An Efficient Interface to Compiled and Fortran Code Supporting Long Vectors
    by Florian Gerber, Kaspar Mösinger and Reinhard Furrer
    December 30, 2023
======================================================================================



 
The  functions .C() and .Fortran() can be used to call compiled and Fortran code from . 
This so-called is convenient, since it does not require any interactions with the C API of .
However, it does not support (, vectors of more than 2^31 elements).
To overcome this limitation, the  package dotCall64 provides ,
which can be used to call compiled and Fortran functions.
It transparently supports and does the necessary castings to pass vectors to arguments of the compiled code.
Moreover, features a mechanism to avoid unnecessary copies of function arguments,
making it efficient in terms of speed and memory usage. 




§ INTRODUCTION

The interpreted character of makes it a convenient front-end for a wide range of applications. 
Although provides a rich infrastructure, it can be advantageous to extend programs with compiled code written in or Fortran <cit.>. 
According to <cit.>, reasons for such an extension are 
the access to new and trusted computations, the increase in computational speed, and the object referencing capabilities. 
For completeness, we also list the reasons against such an extension, which include
an increased workload to write, maintain, and debug the software, platform dependencies, and a less readable source code.

provides two types of interfaces to call compiled code documented in “Writing Extensions” <cit.>.
First, the  feature the functions .Call() and .External(). 
It enables accessing, modifying, and returning objects from using the C API of <cit.>. 
On one hand, this is convenient when the code is specifically written to be used with .
In that case, the C API serves as a glue between and , providing some functionality and control over copying objects on the level. 
On the other hand, it requires the user to learn the C API of .
Especially, when an interface is built on top of existing code this constitutes an additional effort.
Since has no Fortran API, the are not suitable to embed Fortran code into .
Second, the  provides the functions .C() and .Fortran().
This interface allows the compiled code to read and modify atomic vectors, 
which are exposed as the corresponding and Fortran types, respectively.
Thus, no additional API is required, making it favorable for embedding and Fortran code that is not specifically designed for .

On top of these interfaces provided by ,  packages exist that simplify the integration of compiled code into .
One such  package is inline <cit.>, which allows the user to dynamically define  functions and S4 methods with inlined compiled code. 
Other examples are Rcpp <cit.> and its extensions RcppArmadillo <cit.>, RcppEigen <cit.>, RcppParallel <cit.>, and
Rcpp11 <cit.>, which greatly simplify the extension of with C++ code. 
Similar to the , the Rcpp package family is designed to extend with compiled code that is specifically written for that purpose. 

Building packages is a way to share compiled code across different platforms. 
(See, ,  for comments on including portable C++ code in packages.)
As of , 2'303 of the 9'079  packages on CRAN (<http://www.cran.r-project.org/>) include compiled andor Fortran code using both the and the with a similar frequency.
Figures <ref> gives an overview of the number of packages using .C(), .Fortran(), .Call(), and .External(). 



In the remainder of this article, we focus on the intention to embed compiled code into without using its C API.
An example of an  package using that type of interface is the SPArse Matrix package spam <cit.>, which is built around the Fortran library SPARSKIT <cit.>.
Here, the function .Fortran() from the seems to be suitable.
Conversely, using the is also possible but requires adding an additional layer of C code to enable communication between and the compiled Fortran code. 
However, using .Fortran() is also not satisfying, since it lacks flexibility and performance, as also stated in its help page:
“These functions [.C() and .Fortran()] can be used to make calls to compiled C and Fortran 77 code. 
Later interfaces are ‘.Call’ and ‘.External’ which are more flexible and have better performance.”
Two of the missing features of the are:


  * support of ,

  * a mechanism to avoid unnecessary copies of vectors.

The latter is the reason for the lower performance of the compared to the . 
Since the does not allow vectors to be passed to compiled code by reference (without copying),
it is especially impractical for big data application.
The missing features of the motivated the development of the  package dotCall64 <cit.>, which is presented in this article. 





§ LIMITATIONS OF THE 

To set the scene for dotCall64, we first discuss some limitations of the and give insights into the implementation of . 



 §.§ Long vectors

The does not support ; see help("long vector"). 
To understand why extending it to support is a non-trivial task, 
we give more details on the implementation of .
In , vectors are one of the most basic object types underlying more complex objects, such as matrices and arrays. 
They can be thought of as strings of elements that can be indexed according to their relative positions. 
Prior to version 3.0.0, the length of vectors was limited to 2^31-1 elements and indexing thereof was exclusively based on vectors of type integer.
More precisely, the latter are signed vectors having a value range of [-2^31+1, 2^31-1]. 
Starting from the release of version 3.0.0 in early 2013, support for so-called was supplied. 
That is, atomic (, , , , , and ) vectors, , and can now have up to 2^52 elements. 
The introduction of was done with minimal changes in and especially, without changing or adding a data type. 
Vectors of lengths less than 2^31-1 remain unchanged and addressing elements thereof still uses vectors of type .
In contrast, use vectors of type to address elements, which are integer precise up to 2^52.
This implied changes in some functions, such as length(), which returns an or a type depending on whether the input vector is a .
 
> typeof(length(integer(1)))
[1] "integer"
> typeof(length(integer(2^31)))
[1] "double"

Note that as.numeric() returns a type and as.integer() returns an type, 
though both and type are of class "numeric",
see the “Note on names” section in the help page help("is.double"). 

While the implementation of favors backwards compatibility,
care is needed when manipulating those with compiled code. 
We distinguish between passing and indexing :
The former requires passing vectors of more than 2^31-1 elements to complied code and is trivial. 
The latter is challenging, since the indexing vector is of type , whereas the compiled code would naturally expect a type. 
To overcome this discrepancy, one needs to cast the indexing vector from a to a type before calling the compiled code and back-cast it afterwards. 





  
Technical note:
This section gives technical insights into the underlying C implementation of in and may be skipped without loss of the general idea. 
We refer to the source code of version 3.3.1 in several places and show relevant parts thereof in the appendix.
Information on the current and future directions of and types in can be found in “R Internals” <cit.>.

In , vectors are made out of a header of type VECSEXP that is followed by the actual data (Listing <ref>, line 272).  
The header contains a field length of type R_len_t, which is defined as signed (a ). 
Thus, that length field cannot capture the length of a .  
Instead, it is set to -1 whenever the length of the vector is larger than 2^31 - 1, 
and an additional header of type R_long_vec_hdr_t is prefixed.
The prefixed header has a field length of type , which is defined as ptrdiff_t type 
(Listing <ref>, line 75) being “[...] the signed type of the result of subtracting two pointers. 
This will probably be one of the standard signed types (short int, int or long int), but might be a nonstandard type that exists only for this purpose” <cit.>.   

This implementation has the advantage that the existing code does not need to be changed and still works with vectors having less than 2^31 elements.
Hence, the C code of can be changed successively to support throughout several versions, as opposed to changing the entire C code in one step.   
To make C code compatible with , adaptations are needed. 
For example, the widely used C function R_len_t length(SEXP s) (Listing <ref>, line 124) returns the length of a SEXP (S expression) as a R_len_t.
Thus, all instances of that function have to be replaced with calls to the counterpart (, the function R_xlen_t xlength(SEXP s) given in line 159 of Listing <ref>).




 §.§ Copying arguments

The exposes pointers to vectors to compiled code.
In order to avoid any corruption of vectors, they are copied and the compiled code receives pointers to copies of the vectors. 
One exception is when the vector has the named status 0 (, the object is not bound to any symbol); see “Writing Extensions” <cit.>.
This is the case when the passed vector is an evaluated constructor (, integer(1)).
This is often used when the only purpose of the vector is to capture results from the compiled code.

Another situation in which there is no need for copying vectors is when the compiled code only reads an vector without modifying it. 
However, the does not allow the user to avoid copying of vectors (with named status 1 or 2), which leads to a significant computational overhead, especially for large vectors.
Note that prior to version 3.2.0, the copying of vectors could be avoided by setting the argument DUP of .C() and .Fortran() to FALSE.
In later versions, this argument is depreciated and users are referred to the as a more flexible interface; see help(".C") and “R NEWS” <cit.>.




§ THE  PACKAGE DOTCALL64

The limitations of the discussed above have motivated the development of the  package dotCall64.
Its main function is , which can be used to interface compiled code.
In contrast to .C() and .Fortran(), 
it supports and arguments of complied and provides a mechanism to control duplication of function arguments. 
Emphasis was put on providing a trustworthy implementation featuring 
structured and C source code, documentation, examples,
unit tests implemented with testthat <cit.>, 
and scripts containing the later presented performance measurements.





 §.§ Usage of the function 

The function can be used as an enhanced replacement of the and is equally easy to use;
see also the documentation in the reference manual <cit.>.
Its syntax resembles that of the function .C(), and both functions have common arguments as shown in Table <ref>.


The required arguments of are:


  .NAME The name of the compiled function or Fortran subroutine. 

  ... Up to 65 vectors to be accessed by the compiled code.

  SIGNATURE A character vector of the same length as the number of arguments of the . 
Each string specifies the signature of one such argument. 
Accepted signatures are "integer", "double", and "int64". 
The , , and Fortran types corresponding to these specifications are given in Table <ref>.

With that, the following call to the compiled C function void get_c(double input, int index, double output) using .C() can be replaced by its counterpart. 
Therefore, for example, 

> .C("get_c", input = as.double(1:10), index = as.integer(9), output = double(1))

becomes

> .C64("get_c", SIGNATURE = c("double", "integer", "double"), 
+      input = 1:10, index = 9, output = 0)

While more detailed code examples are given later, 
this is enough to highlight some features of .
First, does require the additional argument SIGNATURE specifying the argument types of the . 
In return, it coerces the provided vectors to the specified signatures making the as.double() and as.integer() statements unnecessary. 
Second, all provided arguments can be .
Third, if one of the arguments of the compiled function is a (int64_t in the case of functions, and integer (kind = 8) types for Fortran subroutines), it is enough to set the corresponding SIGNATURE argument to "int64" to successfully evaluate the function.  
That is, does the necessary to and to castings before and after evaluating the compiled code, respectively.

Additional arguments of are the following:


  INTENT A character vector of the same length as the number of arguments of the . 
Each string specifies the intent of one such argument. 
Accepted intents are "rw" (read and write), "r" (read), and "w" (write). 

  NAOK A logical flag specifying whether the vectors passed though ... are checked for missing and infinite values. 

  PACKAGE A character vector of length one restricting the search path of the to the specified package. 

  VERBOSE If 0 (default), no warnings are printed. If 1 and 2, then warnings for tuning and debugging purposes are printed. 

A complete list of arguments including their default values is also given in Table <ref>.

The argument INTENT influences the copying of vectors and can be seen as an enhanced version of the depreciated DUP argument of .C().
By default, all intents are set to “read and write” implying that the compiled code receives pointers to copies of the vector given to ....
This behavior is desirable when the compiled function reads the corresponding vectors and modifies (writes to) them.
For arguments of the that are only read and not modified, the intent can be set to “read.”
With that, the compiled code receives pointers to the corresponding vectors itself.
While this avoids copying, it is absolutely necessary that the compiled code does not alter these vectors, 
as this corrupts the corresponding vectors in the current session. 
For arguments that are only used to write results into it, the intent “write” is suitable. 
To obtain the desired performance gain, the corresponding vectors passed to ... have to be of class "vector_dc".
objects of that class contain information on the type and length of the vectors.
They can be constructed with the function vector_dc(), taking the same arguments as vector() from the base  package. 
For example, instead of passing the vector vector(mode = "numeric", length = 8),
the following object should be passed. 

> vector_dc(mode = "numeric", length = 8)
mode
[1] "numeric"length
[1] 8

attr(,"class")
[1] "vector_dc" "list"     

Based on this information, allocates the corresponding vector (initialized with zeros). 
That vector is then exposed to the compiled function to write into it.
Note that specifying the suitable intent may reduce computation time by avoiding unnecessary copying of vectors
and by avoiding unnecessary to and to castings for SIGNATURE = "int64" type arguments. 
More details on the other arguments are given in the package manual of dotCall64 <cit.>. 




 §.§ Implementation of the function 

The function uses the function .External() from the to directly pass all provided arguments to the C function dC64().
After basic checks of the provided arguments, the function proceeds as schematized in Figure <ref>.
Note that the flowchart depicts the procedure for the case in which the has only one argument. 
Otherwise, dC64() repeats the depicted scheme for all arguments. 



One aspect to highlight is the castings of vectors for SIGNATURE = "int64" arguments. 
For such arguments, the to casting is done for the intents “read and write” and “read”;
see the boxes labeled with (a).
In that case, duplication is not necessary, as the implemented casting allocates a new vector anyway.  
The back-casting from to is only done for the intents “read and write” and “write”;
see the box labeled with (b).

Moreover, an argument of SIGNATURE different from "int64" with intent “read and write” is duplicated in any case; see boxes labeled with (c).
If the intent is “read,” it is not duplicated, 
and if the intent is “write,” the argument is only duplicated when it has a reference status different from 0.
vectors increase their reference status when they are passed to an function, and 
therefore a safe way to allocate a zero initialized vector without copying is to pass an object of class "vector_dc".  

As casting is an expensive operation in terms of computational time, we distribute this task to multiple threads using openMP, if available <cit.>.
Note that the number of used threads can be controlled with the function omp_set_num_threads() from the package OpenMPController <cit.>.
The package dotCall64 can also be compiled without the openMP feature by removing the flag $(SHLIB_OPENMP_CFLAGS) in the src/Makevars file of the source code.





§ EXAMPLES

We showcase the function from the  package dotCall64 with an example function implemented in C and Fortran.
Besides the calls thereof via , the C and Fortran function definitions 
and the commands to compile and load the code are given.
A direct comparison with .C() shows the limitations of the and 
that it is straight forward to overcome these with . 
Moreover, the similarities and differences in the syntax become visible. 
The considered example function takes the arguments input (), index (), and output () 
and writes the element of input at the position specified with index to output.



 §.§ Interface code

A C implementation of the described example function is given next.

void get_c(double *input, int *index, double *output) 
    output[0] = input[index[0] - 1];


We write the function into get_c.c and compile it with the command line command R CMD SHLIB get_c.c.
The resulting dynamic shared object (get_c.so on our Linux platform) must be loaded into before 
the compiled function can be called.
Note that, in the following code, the extension of the shared object is replaced with .Platform$dynlib.ext to make the code platform independent.

> dyn.load(paste0("get_c", .Platformdynlib.ext))

One can use the  to call this function. 
We use the  functions as.double() and as.integer() to ensure that the types of the passed  vectors
match the signature of the C function get_c(). 

> .C("get_c", input = as.double(1:10), index = as.integer(9), output = double(1))output
[1] 9

Next, we try to use the same call with a x_long passed to the argument input of get_c().
 
> x_long <- double(2^31); x_long[9] <- 9; x_long[2^31] <- -1
> .C("get_c", 
+    input = as.double(x_long), index = as.integer(9), output = double(1))output
Error: long vectors (argument 1) are not supported in .Fortran

As expected, .C() throws an error because it does not support . 
The error—and the confusing error message referring to .Fortran() instead of .C()—can 
be avoided by replacing .C() with .
This allows the evaluation of the C function get_c() with the x_long.
Additionally,  requires the argument SIGNATURE encoding the signatures of the arguments of get_c(). 
This information is used to coerce all provided  vectors to the specified signatures.
Thus, it is no longer necessary to reassure that the types of the passed  vectors
match the signature of the compiled function.
 
> install.packages("dotCall64")
> library("dotCall64")
> .C64("get_c", SIGNATURE = c("double", "integer", "double"),
+      input = x_long, index = 9, output = double(1))output
[1] 9

In contrast to the call using .C(), the ninth element of the x_long is returned. 
However, the argument index of get_c() is of type ∫ (a ), 
and hence, elements at positions beyond2^31-1cannot be extracted.
To overcome this, we adapt the definition of the C function get_c() and replace  
the ∫ type in the declaration of the argument index with the  type, 
which is defined in the C header file stdint.h. 

#include <stdint.h>
void get64_c(double *input, int64_t *index, double *output) 
    output[0] = input[index[0] - 1];

We write the function into get64_c.c and compile it with R CMD SHLIB get64_c.c to obtain the dynamic shared object (get64_c.so on our platform).
Because of the  argument, it is not possible to call this function with .C(). 
On the other hand,  can interface this function 
when the second element of the SIGNATURE argument is set to "int64".

> dyn.load(paste0("get64_c", .Platformdynlib.ext))
> .C64("get64_c", SIGNATURE = c("double", "int64", "double"),
+      input = x_long, index = 2^31, output = double(1))output
[1] -1

In the call above, the function  casts the argument index from  (the  representation of ) into a  type vector before calling get64_c(), 
and back-casts it from  to  afterwards. 



 §.§ Interface Fortran code
 
The function  can also be used to interface compiled Fortran code. 
To highlight some Fortran specific features, we translate the C function get_c() into the Fortran subroutine get_f().
 
      subroutine get_f(input, index, output)
      double precision :: input(*), output(*)
      integer :: index    
      output(1) = input(index)
      end
 
Note that we only use lower case letters in the Fortran function and variable names to avoid unnecessary symbol-name translations.
We write the function into the get_f.f and compile it with R CMD SHLIB get_f.f to obtain the dynamic shared object (get_f.so on our platform).
In contrast to .Fortran(),  allows passing pointers to . 

> dyn.load(paste0("get_f", .Platformdynlib.ext))
> .C64("get_f", SIGNATURE = c("double", "integer", "double"),
+      input = x_long, index = 9, output = double(1))output
[1] 9

Again, elements with positions beyond2^31-1cannot be accessed, 
since the argument index is of type  and compiled as a  by default. 
To make get_f() compatible with , we can either change the declaration of index to integer (kind = 8) index in get_f.f or 
leave the Fortran code unchanged and set the following compiler flag to compile  as .

MAKEFLAGS="PKG_FFLAGS=-fdefault-integer-8" R CMD SHLIB get_f.f 

Note that both the kind = 8 declaration and the -fdefault-integer-8 flag are valid for the GFortran compiler <cit.> and may not have the intended effect using other compilers.
The resulting dynamic shared object from the command above (get_f.so on our platform) can be called from  as follows.

> dyn.load(paste0("get_f", .Platformdynlib.ext))
> .C64("get_f", SIGNATURE = c("double", "int64", "double"),
+      input = x_long, index = 2^31, output = double(1))output
[1] -1


 §.§ Extend  packages to support long vectors

Extending  packages to support  allows developers to distribute compiled code featuring  with an  user interface.
Given the popularity of , this is a promising approach to make such software available to many users.
With the function , the workload of extending an  package to support  is reduced to the following tasks:

  *  replace the R function to call compiled code with ,

  *  replace the  type declarations in the compiled code with a  declaration.

The latter task implies replacing all ∫ type declarations in  code with int64_t type declarations
and replacing all  type declarations in Fortran code with integer (kind = 8). 
In both cases, the replacements can be automatized (, with the stream editor ).
If the considered Fortran code does not explicitly declare the bits of the integers, 
an alternative approach is to set the compiler flag -fdefault-integer-8 to compile integers as  using GFortran compilers. 
This is convenient because the Fortran code does not need to be changed at all in that case. 

A more elaborate extension could feature two versions of the compiled code:
one with  and the other one with . 
Then, the  function can dispatch to either version according to the sizes of the involved vectors. 
This avoids  to  castings when only vectors with less than2^31-1elements are involved. 
It is convenient to manage two versions of compiled code by putting them into two separate R packages. 
The first package includes the compiled code with  together with the  code and the documentation. 
This package can be used independently as long as no  are involved. 
The second package can be seen as an add-on package and includes only the compiled code with integers declared as . 
Thus, loading both packages enables  support. 
This separation into two packages has the advantage that the compiled functions featuring  and their  counterparts can have the same name.
The desired function is then specified by setting the appropriate PACKAGE argument of .

As a proof of concept, we extended the sparse matrix algebra  package spam to handle 
sparse matrices with more the2^31-1non-zero elements. 
From the user perspective, the syntax to manipulate such matrices remains the same.
In fact, spam users may not even notice the extension.
In the case, in which the number of non-zero entries of a matrix exceeds2^31-1and 
the add-on package spam64 is loaded,
spam automatically dispatches to the compiled code with . 
The new capabilities of spam and spam64 were illustrated with a parametric model of a non-stationary spatial covariance matrix fitted to satellite data.
More information on spam64 and the data example is given by <cit.>. 




§ PERFORMANCE

There are different settings in which the elapsed time to interface compiled code is relevant.
One of those is when the compiled code is interfaced often and takes only a short time to evaluate. 
Here, the overhead of the interface becomes relevant, which is in the order of a few microseconds for .
Another such setting is when large and possibly  are passed through .
In that case, the overhead is negligible, as other services of the interface and the execution of the compiled code take up several orders of magnitude more time.
When  is used to interface  arguments of the compiled code, 
the largest share of the elapsed time is caused by the  to  and  to  castings. 
Since castings are implemented with openMP, the elapsed time thereof also depends on the number of used threads. 
Besides that, copying objects and checking them for missing infinite values are also time-consuming operations. 

Another performance aspect is peak memory usage. 
Using the default arguments of , its peak memory usage is about twice the size of the  vectors passed through ..., and hence, is similar to .C(). 
An exception where the peak memory usage is reduced is indicated below. 



 §.§ Performance relevant arguments of 

Further,  provides arguments to optimize calls to compiled code,
one of which is the argument INTENT, which is set to “read and write” by default.
Since many  only read or write to certain arguments, it is safe to avoid copying in some cases. 
For example, the C function get64_c(), as defined above, only reads the arguments input and index and only writes to the argument output.
Thus, we can set the INTENT argument of  to c("r", "r", "w") and pass the argument with intent “write” as objects of class "vector_dc" to reduce the copying of  vectors to a minimum.
Another significant performance gain is obtained by setting the argument NAOK to TRUE.
This avoids checking the  vectors passed through ... for NA, NaN, and Inf values. 
Small-scale performance gains can be achieved by setting the PACKAGE argument, which reduces the time to find the compiled code, and by setting VERBOSE= 0, which avoids the execution of getOptions("dotCall64.verbose"). 
Similar speed considerations that are partially applicable to  are given in “Writing  Extensions”<cit.>.
An optimized version of the call to the C function get64_c(), taking the discussed performance considerations into account, is given next.

> .C64("get64_c", SIGNATURE = c("double", "int64", "double"),
+      input = x_long, index = 2^31, output = numeric_dc(1),
+      INTENT = c("r", "r", "w"), NAOK = TRUE, PACKAGE = "dotCall64", VERBOSE = 0)


 §.§ Timing measurements

In the following, we present detailed timing measurements and benchmark  against .C(), where possible. 
We consider the following C function contained in the  package dotCall64.
 
void BENCHMARK(void *a)  
This function takes one pointer a to a variable of an unspecified data type and does no operations with it. 
Thus, the elapsed time to call BENCHMARK() from  is dominated by the performance of the used interface.
We measure the time to call this function with different NAOK and INTENT settings of  and benchmark it against .C() using microbenchmark<cit.>. 
To get an estimate of the measurement uncertainty, we repeated the measurements between100and10'000times and report the median elapsed time as well as the interquartile range (IQR) of the replicates.  
Naturally, timing measurements are platform dependent. 
We produced the presented results on Intel Xeon CPU E7-2850 2.00 GHz processors using a  Linux environment where  was installed with default installation flags. 
When not indicated differently, the measurements were produced using a single thread. 

First, we consider the situation in which a pointer to an  vector of length one is passed to the compiled C function BENCHMARK().
The following truncated  code illustrates how the measurements were performed.
The complete  scripts implementing all presented performance measurements are available in the benchmark directory in the source code of dotCall64. 
 
> library("microbenchmark")
> int <- integer(1)
> microbenchmark(
+   .C("BENCHMARK", a = int, NAOK = FALSE, PACKAGE = "dotCall64"),
+   .C64("BENCHMARK", SIGNATURE = "integer", a = int, INTENT = "rw", 
+        NAOK = FALSE, PACKAGE = "dotCall64", VERBOSE = 0),
+   .C64("BENCHMARK", SIGNATURE = "integer", a = int, INTENT = "r",  
+        NAOK = FALSE, PACKAGE = "dotCall64", VERBOSE = 0),
...

Since the  vector int is very short, 
a large part of the elapsed time in this experiment is caused by the overhead of the interfaces. 
Table <ref> presents the resulting timing measurements in microseconds. 
They indicate that .C() is more than two times faster compared to . 
However, this is not surprising, since  is more flexible and therefore has a larger overhead. 
The arguments NAOK and INTENT have little influence on the elapsed times.
The IQRs of around one microsecond indicate a relatively large variability of the elapsed time, which is typical for short timing measurements.



We repeated the same experiment with vectors of length2^28. 
Now, the elapsed times are dominated by services of the interfaces (, checking for missing infinite values, copying, and casting).
The timings in seconds are presented in Table <ref>. 
They indicate that  with argument INTENT= "rw" and .C() showed similar elapsed times.
When the intent is set to “read” (INTENT= "r"), the elapsed times were reduced and dropped to0.00seconds for some configurations. 
Moreover, not checking for missing infinite values (NAOK= TRUE) decreases the elapsed times across all considered cases.
The castings of SIGNATURE= "int64" arguments seems to be the most time-consuming task. 
Note that the IQRs are now smaller relative to the measured timings, because the measured times are larger.




In a second series of timing measurements, we consider the situation in which a pointer to a vector is passed to the compiled code to write into the vector.
We measure the elapsed times of this task as shown in the following truncated  code.

> microbenchmark(
+   .C("BENCHMARK", a = integer(2^28), NAOK = TRUE, package = "dotCall64")
+   .C64("BENCHMARK", SIGNATURE = "integer", a = integer(2^28), INTENT = "rw", 
+        NAOK = TRUE, package = "dotCall64", VERBOSE = 0)
+   .C64("BENCHMARK", SIGNATURE = "integer", a = integer_dc(2^28), INTENT = "w", 
+        NAOK = TRUE, package = "dotCall64", VERBOSE = 0),
...

Note the usage of integer_dc(), which creates a list containing the length and class of the vector.
This information is then used by  to create the corresponding vector in C.
Table <ref> shows the timing measurements for the described setting.
As expected using  with INTENT= "w" reduces the elapsed times 
compared to INTENT= "rw" substantially. 
Furthermore, .C() and   with INTENT= "w" have similar elapsed times.
While .C() relies on the reference counting mechanism of  objects to avoid copying <cit.>,  uses the "vector_dc" class.
The latter has the advantage that one  to  casting can be avoided in the SIGNATURE= "int64" case. 



The function  features an openMP implementation of the  to  and  to  castings of
SIGNATURE="int64" arguments.
Hence, the computational workload of the castings can be distributed to several threads running in parallel.
To quantify the performance gain related to using openMP, 
we control the number of used threads to be between1and10with the  package OpenMPController and measure the elapsed times of the following call.
 
> .C64("BENCHMARK", SIGNATURE = "int64", a = a, INTENT = "rw", NAOK = TRUE,  
+      PACKAGE = "dotCall64", VERBOSE = 0)

We let a be  vectors of length2^16,2^22,2^28, and2^34and performed five replicated timing measurements for each configuration.
The results are summarized in Figure <ref>.
The reduction in computation time due to using multiple threads is greatest for the vectors of length2^34, where using10threads reduced the elapsed times by about70%.
Conversely, for the vector of length2^16no reduction was observed.



§ SUMMARY

This paper presents the  package dotCall64, which provides an alternative to .C() and .Fortran() from the . 
In the first section, we introduce 's interfaces to embed compiled  and Fortran code. 
We argue that, in some situations, a .C() type interface is more convenient compared to using the C API of  in conjunction with the . 
In section two, we motivate the development of dotCall64 with a discussion of missing features of the  and an overview of the  implementation of .
Then, we present the usage and the implementation of the  function from the  package dotCall64.
This is followed by examples demonstrating the capabilities of the new interface—also in comparison with the . 
Furthermore, we discuss strategies to extend entire  packages with compiled code supporting . 
In the last section, we present performance measurements of the  interface and benchmark it against .C(). 
This highlights the speed gains achieved by avoiding unnecessary copies of  vectors and by using openMP for casting  vectors. 
In conclusion, the interface provided by the  package dotCall64 is an up-to-date version of the  including tools to conveniently embed compiled code manipulating . 



§ ACKNOWLEDGMENTS

We thank Rafael Ostertag for contributions to Figure <ref>. 
We acknowledge the support of the University of Zurich Research Priority Program (URPP) on “Global Change and Biodiversity.”

Florian Gerber

Department of Mathematics

University of Zurich

8057 Zurich
 
Switzerland
florian.gerber@math.uzh.chKaspar Mösinger

Department of Mathematics

University of Zurich

8057 Zurich
 
Switzerland
kaspar.moesinger@gmail.comReinhard Furrer

Department of Mathematics and 

Department of Computational Science

University of Zurich

8057 Zurich
 
Switzerland
reinhard.furrer@math.uzh.ch

§ APPENDIX: R SOURCE CODE

In the following, we show parts of the C source code of  version 3.3.1 to support the understanding of the  implementation. 
More precisely, the lines 26–377 from the file Rinternals.h and the lines 124–191 from the file Rinlinedfuns.h are shown in Listing <ref> and Listing <ref>, respectively. 
The indicated line numbers in the code refer to the actual line numbers of the corresponding file.  

[firstnumber=26,firstline=26,lastline=377,label=list:Rinternals,caption=R-3.3.1/src/include/Rinternals.h]Rinternals.h[firstnumber=124,firstline=124,lastline=191,label=list:Rinlinedfuns,caption=R-3.3.1/src/include/Rinlinedfuns.h]Rinlinedfuns.h