title: How to Configure Chapel/GASNet for running multilocale codes on MXM Infiniband network with Partition Key?
body: <p>I'm trying to run a multilocale Chapel code on a cluster that has an MXM Infiniband network(40 Gbps, model: Mellanox Technologies MT26428).  </p>
  
  <p>I followed both Chapel and GASNet documentations, and I set</p>
  
  <p><code>export CHPL_COMM_SUBSTRATE=ibv</code></p>
  
  <p><code>export CHPL_LAUNCHER=gasnetrun_ibv</code></p>
  
  <p><code>export GASNET_IBV_SPAWNER=mpi</code></p>
  
  <p>instead of using <code>CHPL_COMM_SUBSTRATE=mxm</code>, once mxm is deprecated.</p>
  
  <p>The problem is that I can build Chapel using the ibv substrate. But, I cannot run on multiple locales. I receive a huge number of timeout errors. </p>
  
  <p>In the first place, I thought the problem was the PKEY. So, I added <code>"--mca btl_openib_pkey "0x8100""</code> to the <code>MPIRUN_CMD</code>. But, no success.</p>
  
  <p>I also tried to use the deprecated mxm configuration:</p>
  
  <p><code>CHPL_LAUNCHER=gasnetrun_mxm</code></p>
  
  <p><code>export CHPL_LAUNCHER=gasnetrun_ibv</code></p>
  
  <p><code>export GASNET_MXM_SPAWNER=mpi</code></p>
  
  <p>However, I cannot build Chapel with such a configuration. That's the error message:</p>
  
  <p><code>"User requested --enable-mxm, but I don't know how to build mxm programs for your system."</code></p>
  
  <p>By the way, using GASNET on top of MPI, UDP, and Infiniband without a Partition Key works just fine.</p>
  
  <p>Does anybody know how to use Chapel on a Cluster equipped with an MXM Infiniband network and Partition Key (PKEY)?</p>
  
  <p>Best Regards,</p>
  
  <p>Tiago Carneiro.</p>