mpiio.cpp File Reference

Concerning the file layouts. More...

#include "mpiio.hpp"
#include "Particle.hpp"
#include "bonded_interactions/bonded_interaction_data.hpp"
#include "cell_system/CellStructure.hpp"
#include "errorhandling.hpp"
#include "system/System.hpp"
#include <utils/Vector.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/device/back_inserter.hpp>
#include <boost/iostreams/stream.hpp>
#include <mpi.h>
#include <algorithm>
#include <cassert>
#include <cerrno>
#include <cstddef>
#include <cstdio>
#include <cstring>
#include <sstream>
#include <string>
#include <sys/stat.h>
#include <tuple>
#include <utility>
#include <vector>

Include dependency graph for mpiio.cpp:

Go to the source code of this file.

Namespaces
namespace	Mpiio

Functions
static bool	Mpiio::fatal_error (char const *msg, std::string const &fn="", std::string const &extra="")
	Fatal error handler.
static bool	Mpiio::fatal_error (char const *msg, std::string const &fn, MPI_File *fp, int errnum)
	Fatal error handler that closes an open file and queries the message associated with an MPI error code.
template<typename T >
static void	Mpiio::mpiio_dump_array (const std::string &fn, T const *arr, std::size_t len, std::size_t pref, MPI_Datatype MPI_T)
	Dump data arr of size len starting from prefix pref of type T using MPI_T as MPI datatype.
static unsigned long	Mpiio::mpi_calculate_file_offset (unsigned long n_items)
	Calculate the file offset on the local node.
static void	Mpiio::dump_info (std::string const &fn, unsigned fields, BondedInteractionsMap const &bonded_ias)
	Dump the fields and bond information.
void	Mpiio::mpi_mpiio_common_write (std::string const &prefix, unsigned fields, BondedInteractionsMap const &bonded_ias, ParticleRange const &particles, write_buffers &buffers)
	Parallel binary output using MPI-IO.
static unsigned long	Mpiio::get_num_elem (const std::string &fn, std::size_t elem_sz)
	Get the number of elements in a file by its file size and elem_sz.
template<typename T >
static void	Mpiio::mpiio_read_array (const std::string &fn, T *arr, std::size_t len, std::size_t pref, MPI_Datatype MPI_T)
	Read a previously dumped array of size len starting from prefix pref of type T using MPI_T as MPI datatype.
static unsigned	Mpiio::read_head (const std::string &fn, int rank)
	Read the header file and return the first value.
static std::tuple< unsigned long, unsigned long >	Mpiio::read_prefs (const std::string &fn, int rank, int size, unsigned long nglobalpart)
	Read the pref file.
void	Mpiio::mpi_mpiio_common_read (std::string const &prefix, unsigned fields, CellStructure &cell_structure)
	Parallel binary input using MPI-IO.

Detailed Description

Concerning the file layouts.

• Scalar arrays are written like this: rank0 — rank1 — rank2 ... where each rank dumps its scalars in the ordering of the particles.
• Vector arrays are written in the rank ordering like scalar arrays. The ordering of the vector data is: v[0] v[1] v[2], so the data looks like this: v1[0] v1[1] v1[2] v2[0] v2[1] v2[2] v3[0] ...

To be able to determine the rank boundaries (a multiple of nlocalparts), the file 1.pref is written, which dumps the partial sum of nlocalparts, i.e. the prefixes in scalar arrays:

• 1.prefs looks like this: 0 nlocalpats_rank0 nlocalparts_rank0+nlocalparts_rank1 ...

Bonds are dumped as two arrays, namely 1.bond which stores the bonding partners of the particles and 1.boff which stores the iteration indices for each particle.

• 1.boff is a scalar array of size (nlocalpart + 1) per rank.
• The last element (at index nlocalpart) of 1.boff's subpart [rank * (nlocalpart + 1) : (rank + 1) * (nlocalpart + 1)] determines the number of bonds for processor rank.
• In this subarray one can find the bonding partners of particle id[i]. The iteration indices for local part of 1.bonds are: subarray[i] : subarray[i+1]
• Take a look at the bond input code. It's easy to understand.

Definition in file mpiio.cpp.