Abstract
A simple parallelization approach using General Purpose computation on Graphics Processing Unit was applied for solving the MHD equations using the CIP-MOCCT method. We investigated the efficiency of this parallelization approach and found that the computational speed of the modified code is significantly improved despite the simple modification.
Introduction
A method for the plasma dynamics analysis is numerical simulation by solving the MHD equations. However, solving them in three dimensions is expensive and requires excessive computing time; hence, faster processors are required. Rapid improvement in Central Processing Units (CPUs) over the last several decades has enabled us to run larger and more accurate numerical simulations. The results of these simulations have made great contributions to plasma and fusion research. The field of numerical simulation has a high potential for future growth. Recently, however, there have been few CPU clock speed enhancements. Instead, CPU performance is increased using multiple processors via the multi-core strategy. This indicates that further improvements of computing speed are obtained by parallel computation. Therefore, it will be necessary for us to employ the multi-core processor technology.
Introduction
We are interested in using the Graphics Processing Unit (GPU) as a multi-core processor to perform parallel computations. The GPU is a multi-core processor developed for graphics processing and affords massive parallel computations. In recent years, the high parallel performance of the GPU has motivated research outside graphics processing, i.e., general purpose computation; such techniques are called General Purpose computation on GPU (GPGPU).
Introduction
GPGPU has been used in some simulation studies. In fluid dynamics, Brandvik and Pullan [1] solved twodimensional (2D) and three-dimensional (3D) Euler equations with uniform grids reducing the computational time by a factor of 29 (2D) and 16 (3D). Elsen et al. [2] solved 3D Euler equations on multi-block meshes. In plasma physics, Stantchev et al. [3] discussed the efficient implementation of the particle-in-cell plasma simulation code. Wang et al. [4] solved the MHD equations by using an adaptive mesh finite volume method. However, there are few studies on the application of GPU for solving the full MHD equations. Because the experts in numerical simulations are not always experts in parallel computation, the approach involving GPGPU must be made simple to use. It is important for us to know the efficiency of a simple parallelization approach for solving numerical simulations. In the present study, we apply this approach to solve the MHD equations and investigate its efficiency.
Introduction
For simple parallelization, an explicit scheme is convenient, because the solution of simultaneous equations is unnecessary. We adopt a combination of the Constrained Interpolation Profile (CIP) [5] , Method of Characteristics (MOC) [6] , and Constrained Transport (CT) methods [7] ; we call it the CIP-MOCCT method [8] , which is an explicit scheme. The numerical phase errors of the advection equation solved by the CIP method are smaller than those produced by other schemes [9] . In addition, the CIP method can accurately follow the contact discontinuity and the boundary between the plasma and vacuum. This method has been used in a number of hydrodynamic [10] and MHD simulations [11, 12] . However, as mentioned in [13] , the application of the CIP method to the MHD problems is difficult because of the traverse mode Alfvén wave and the divergence-free condition of a magnetic field. Thus, a special treatment of the Alfvén wave and the divergence-free condition is required. The MOCCT method is a standard technique for treating the Alfvén wave and for satisfying the divergence-free condition. The CIP-MOCCT method is not used widely in the field of fusion science, but it has been employed in several astrophysics studies [14] [15] [16] [17] [18] [19] . To the best of our knowledge, this is the first attempt to apply GPGPU to the CIP-MOCCT method.
GPGPU and CUDA
Although performance improvements using GPGPU have been studied [20] , GPU is essentially the hardware for graphics processing. In early studies employing GPGPU, programmers had to apply their algorithms and data structures to the graphics programming model, which was not always straightforward, and hence, in-depth knowledge of graphics programming was required.
GPGPU and CUDA
Compute Unified Device Architecture (CUDA) [21] is the development environment for GPGPU provided by NVIDIA. as NVIDIA r CUDA TM . CUDA makes the development of GPU computing applications much easier and more efficient than earlier attempts of using GPGPU. CUDA-C is an extension of the C Programming Language and is easy to learn for programmers with knowledge of C.
Implementation of CUDA
To calculate efficiently in CUDA, we have to appropriately use the several types of memories on GPU, e.g., global memory, shared memory, constant memory, texture memory and register. In particular, the register and shared memory accessed via high bandwidths are essential for higher optimization, although these memories are difficult to use efficiently. For this reason, the optimization strategy involves the effectively use of these memories. On the other hand, although the access speed of the global memory is low, it has much larger memory space and is easier to program compared with the shared memory. In this study, a computational code is modified via a simple parallelization approach using only the global memory.
Implementation of CUDA
In a parallelization approach, we need to find bottlenecks in the existing code. Such bottlenecks were found in triple nested loops in our existing 3D MHD simulation code. In CUDA, the simplest parallelization approach for these loops is achieved by mapping each array element to the corresponding thread on the GPU (Fig. 1) . This means that the number of threads invoked on the GPU is equal to the number of array elements. Because the shared memory is not used, this approach is not the best for optimization in terms of the calculation speed. However, it has major advantages: (i) we can easily modify the existing code and (ii) it is applicable to other numerical schemes. List 1 shows the triple nested loop typically used in the 3D finite difference method in the computation on the CPU. In List 1, the elements of the 3D array are calculated while incrementing the indices i, j, and k of the triple nested loop. The modified version of the code in List 1 is shown in List 2. There are three functions func, func kernel, and IDX. func is executed on the CPU and calls func kernel, which is executed on the GPU. Variables BLOCKS and THREADS in parentheses, <<< and >>>, are parameters specifying the number of threads run on the GPU. In the CUDA architecture, threads are arranged into threadblocks. BLOCKS and THREADS indicate the number of threadblocks and the number of threads per threadblock, respectively. The product of BLOCKS and THREADS is the number of the total threads running on the GPU. func kernel is a "kernel function" invoked by the CPU and runs on the GPU. The processes performed in each thread are implemented in this kernel. Such kernel functions require a __global__ qualifier for their definition. At the beginning of this kernel, the thread index idx, which indicates the location of the threads, is calcu-lated using built-in variables blockDim, blockIdx, and threadIdx. The index idx corresponds to the index of the array. We manipulate the 3D array as the 1D array in List 2. The function IDX is invoked to obtain the index of the 1D array from the index of the 3D array. The __device__ qualifier is required for the functions executed on the GPU.
Implementation of CUDA
In the present study, the modifications shown in List 2 have been implemented for all triple nested loops. As a result, these loops can be parallelized to be computed on the GPU.
MHD Equations and Numerical
Scheme in the Developed Code
MHD equations
MHD equations in the developed code are expressed as follows.
MHD equations
Here, ρ is the density, V the velocity, P the pressure, J the current density, Φ the viscosity stress tensor, B the magnetic field, E the electric field, γ the ratio of specific heat and η the resistivity.
CIP method
For fluid dynamics equations (1)- (3), we apply the CIP method [5] , which is an explicit numerical technique applied to various problems, including fluid dynamics. In this method, a value and its derivative are treated as independent physical variables. It enables us to accurately compute phenomena such as shockwaves, whose values change sharply in space.
CT method
The divergence-free condition of magnetic field B (Eq. (7)) does not appear in the time evolution equations (Eqs. (4)- (5)). It is pointed out that even small errors in satisfying the divergence-free condition cause artifacts such as a non-physical force parallel to the magnetic field [22] . In order to satisfy the above condition, the CT method [7] is adopted in the developed code.
CT method
Other schemes such as the projection scheme [22] require a Poisson solver. Therefore, we also have to develop a Poisson solver that can use the GPU. On the other hand, the CT method enables the satisfaction of the divergencefree condition with the accuracy of the finite difference method by only defining V, B, and E on the staggered grid (Fig. 2 ).
MOC method
To compute the time evolution of B by using Eq. (4), it is necessary to calculate E from Eq. (5). But in the staggered grid defined by the CT method, the variables of V and B are not defined at the location of the variables of E. Thus, it is necessary to interpolate the values of V and B at the location of the variables of E. However, it is well known that numerical oscillation in the Alfvén wave propagation is observed when the simple average of V and B values at the location of the variables of E is used. The MOC method [6] is used for the stable computation of the Alfvén wave. From Eqs. (1), (2), and (4)- (6), a set of the characteristic equations are given by,
MOC method
These equations show that V y ∓ B y / √ ρ are constant on the characteristic lines and that the value of V y and B y at E z are obtained. In addition, other values of the components of V and B at the locations of the variables of E are obtained similarly.
Comparison between CPU and GPU
To evaluate the accuracy of the GPU solution, we applied both the CPU and GPU codes to a 1D MHD shock tube problem [23] . The initial condition for this problem was separated into two uniform states at the position of x = 0. set Δx = 1/200, Δt = Δx/10, γ = 2, and η = 0. The results at t = 0.1 are shown in Fig. 3 . It can be seen that the GPU solution is almost the same as the CPU solution. Moreover, these results also coincide with Brio-Wu's results. This proves the validity of our developed code. The performance of the modified 3D MHD simulation code using GPU in double precision was evaluated. The simulation settings were the same as for the abovementioned MHD shock tube problem. We evaluated the CPU time from the start of the time integration (t = 0) until 100 time steps. Computational times using a CPU (Intel r Core TM i7, 2.93 GHz) and two GPUs (NVIDIA r GeForce r GTX 285 and GTX 480) were measured. Although Intel Core i7 is a multi-core processor, the computational time was measured for the case of using a single processor. The specifications of the GPUs are shown in Table 1 .
Comparison between CPU and GPU
The computational times measured for various mesh points are summarized in Fig. 4 . It can be seen that for this calculation GTX285 is about 13 times faster than the CPU. Furthermore, GTX480 is about 30 times faster than the CPU. These results show the effectiveness of our simple parallelization using the GPU. This means that the improvement in speed is independent of the total number of mesh points (NX × NY × NZ). In addition, these results are independent of the problems, because the CIP-MOCCT method is an explicit scheme without a relaxation algorithm. Fig. 4 Computational time using a CPU (Core i7) and two GPUs (GTX285 and GTX480). The horizontal axis is the total number of mesh points (NX × NY × NZ).
Summary and Discussion
We developed an MHD simulation code using the CIP-MOCCT method and modified it to extend its application to the parallel computing code using the GPU. Although we used a simple parallelization approach without utilizing the shared memory, the computational speed of the modified code was improved significantly.
Summary and Discussion
The total memory on a GPU is smaller than that on a CPU. For this reason, increase in the number of mesh points lead to shortage in the GPU memory space. The use of multiple GPUs is considered a solution for this problem. However, the transfer of data between the CPU and the GPU or between computers in the multi GPU technique incurs large costs.

{'paper_id': '121076814', 'title': 'Effectiveness of GPGPU for Solving the Magnetohydrodynamics Equations Using the CIP-MOCCT Method', 's2_url': 'https://api.semanticscholar.org/CorpusID:121076814', 'source': 'S2ORC'}