title: Creating extendable datasets in parallel HDF5
body: <p>I am trying to write data into an hdf5 file in parallel.  Each node has its own dataset, which is unique (although they are the same size).  I am trying to write them all into separate datasets in an hdf5 file in parallel.  The catch is that later I may want to overwrite them with different size datasets (different size compared to the original set of datasets --- the dataset on each processor is the same size).  Does anyone know how to do this?</p>
  
  <p>(code relies on boost and Eigen)</p>
  
  <p>I have code to first open the file:</p>
  
  <pre><code>boost::mpi::environment env(argc, argv);
  
  // set up the info for HDF5 and MPI
  MPI_Comm comm = MPI_COMM_SELF;
  MPI_Info info = MPI_INFO_NULL;
  
  // Set up file access property list with parallel I/O access
  hid_t plist_id = H5Pcreate(H5P_FILE_ACCESS);
  H5Pset_fapl_mpio(plist_id, comm, info);
  
  // declare the file ID
  std::string filename = "test.h5";
  
  // create a file
  hid_t fileID = H5Fcreate(filename.c_str(), H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
  
  // close the property list
  H5Pclose(plist_id);
  </code></pre>
  
  <p>Then to create and write the datasets:</p>
  
  <pre><code>// get the mpi communicator
  unique_ptr&lt;mpi::communicator&gt; worldComm(new mpi::communicator);
  
  const Eigen::VectorXd dataset = worldComm-&gt;rank() * Eigen::VectorXd::Ones(3);
  const std::string name = "/vector";
  
  // sleep for a bit so the processors are doing something different
  sleep(worldComm-&gt;rank() * 2.0);
  
  // the sizes of the data set
  const hsize_t dimsf[2] = {(hsize_t)dataset.rows(), (hsize_t)dataset.cols()};
  
  // set the maximum size of the data set to be unlimited
  const hsize_t maxdim[2] = {H5S_UNLIMITED, H5S_UNLIMITED};
  
  // the size of each chuck --- is there a better way to choose these numbers!?!?!?!
  const hsize_t chunkDims[2] = {2, 5};
  
  // create the dataspace for the dataset.
  const hid_t filespace = H5Screate_simple(2, dimsf, maxdim); 
  assert(filespace&gt;0);
  
  // modify data set creation properties --- enable chunking
  const hid_t prop = H5Pcreate(H5P_DATASET_CREATE);
  const hid_t status = H5Pset_chunk(prop, 2, chunkDims);
  
  // create the dataset with default properties for each process
  std::vector&lt;hid_t&gt; dsetVec(worldComm-&gt;size());
  for( int i=0; i&lt;worldComm-&gt;size(); ++i ) {
    const std::string datasetName = name+"_rank_"+std::to_string(i);
  
    dsetVec[i] = H5Dcreate2(fileID, datasetName.c_str(), H5T_NATIVE_DOUBLE, filespace, H5P_DEFAULT, prop, H5P_DEFAULT);
  }
  
  // Create property list for dataset write.
  const hid_t plistID = H5Pcreate(H5P_DATASET_XFER);
  
  // write the data to file
  H5Dwrite(dsetVec[worldComm-&gt;rank()], H5T_NATIVE_DOUBLE, H5S_ALL, H5S_ALL, plistID, dataset.data());
  
  // close the filespace 
  H5Sclose(filespace);
  
  // close the datasets
  for( int i=0; i&lt;worldComm-&gt;size(); ++i ) {
    H5Dclose(dsetVec[i]);
  }
  
  // close the file
  H5Fclose(fileID);
  </code></pre>
  
  <p>What I expect is four datasets named "/vector_rank_i" (i=0,1,2,3) each of size 3 and valued [0, 0, 0], [1, 1, 1], [2, 2, 2], and [3, 3, 3], respectively.  However, what is being produced is four datasets named "/vector_rank_i" (i=0,1,2,3) each of size 3 but the values are [0, 0, 0], [0, 0, 0], [0, 0, 0], and [3, 3, 3].  </p>
  
  <p>This exact code works perfectly if I don't use chunking.  However, since I'll need to be able to extend the datasets later this is less than ideal.  Does anyone know a good work around?</p>