communication.hpp File Reference

This file contains the asynchronous MPI communication. More...

#include "MpiCallbacks.hpp"
#include <utils/Vector.hpp>
#include <boost/mpi/communicator.hpp>
#include <boost/mpi/environment.hpp>
#include <memory>
#include <utility>

Include dependency graph for core/communication.hpp:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
struct	Communicator
struct	MpiContainerUnitTest

Namespaces
namespace	Communication

Functions
MpiCallbacks &	Communication::mpiCallbacks ()
	Returns a reference to the global callback class instance.
std::shared_ptr< MpiCallbacks >	Communication::mpiCallbacksHandle ()
std::shared_ptr< boost::mpi::environment >	mpi_init (int argc=0, char **argv=nullptr)
	Initialize MPI.
template<class... Args, class... ArgRef>
void	mpi_call (void(*fp)(Args...), ArgRef &&...args)
	Call a local function.
template<class... Args, class... ArgRef>
void	mpi_call_all (void(*fp)(Args...), ArgRef &&...args)
	Call a local function.
void	mpi_loop ()
	Process requests from head node.
void	Communication::init (std::shared_ptr< boost::mpi::environment > mpi_env)
	Init globals for communication.
void	Communication::deinit ()

Variables
int	this_node
	The number of this node.
boost::mpi::communicator	comm_cart
	The communicator.
Communicator	communicator

Detailed Description

This file contains the asynchronous MPI communication.

It is the header file for communication.cpp.

The asynchronous MPI communication is used during the script evaluation. Except for the head node that interprets the interface script, all other nodes wait in mpi_loop() for the head node to issue an action using mpi_call(). mpi_loop() immediately executes an MPI_Bcast and therefore waits for the head node to broadcast a command, which is done by mpi_call(). The request consists of a callback function with an arbitrary number of arguments.

To add new actions (e.g. to implement new interface functionality), do the following:

• write the mpi_* function that is executed on the head node
• write the mpi_*_local function that is executed on worker nodes
• register the local function with one of the REGISTER_CALLBACK macros

After this, your procedure is free to do anything. However, it has to be in (MPI) sync with what your new mpi_*_local does. This procedure is called immediately after the broadcast.

Definition in file core/communication.hpp.

Function Documentation

mpi_call()

template<class... Args, class... ArgRef>

void mpi_call	(	void(*)(Args...)	fp,
		ArgRef &&...	args
	)

Call a local function.

Template Parameters

Args	Local function argument types
ArgRef	Local function argument types

Parameters

fp	Local function
args	Local function arguments

Definition at line 106 of file core/communication.hpp.

References Communication::mpiCallbacks().

Referenced by build_particle_node(), and mpi_get_particles().

mpi_call_all()

template<class... Args, class... ArgRef>

void mpi_call_all	(	void(*)(Args...)	fp,
		ArgRef &&...	args
	)

Call a local function.

Template Parameters

Args	Local function argument types
ArgRef	Local function argument types

Parameters

fp	Local function
args	Local function arguments

Definition at line 117 of file core/communication.hpp.

References Communication::MpiCallbacks::call_all(), and Communication::mpiCallbacks().

mpi_init()

std::shared_ptr< boost::mpi::environment > mpi_init	(	int	argc = 0,
		char **	argv = nullptr
	)

Initialize MPI.

Definition at line 120 of file communication.cpp.

Referenced by MpiContainerUnitTest::MpiContainerUnitTest().

mpi_loop()

void mpi_loop

(

)

Process requests from head node.

Worker nodes main loop.

Definition at line 124 of file communication.cpp.

References Communication::MpiCallbacks::loop(), and this_node.

Variable Documentation

comm_cart

boost::mpi::communicator comm_cart

extern

The communicator.

Definition at line 47 of file communication.cpp.

Referenced by ComFixed::apply(), FFTBackendLegacy< FloatType >::backward_fft(), benchmark_integration_step(), DipolarP3MImpl< FloatType, Architecture >::calc_average_self_energy_k_space(), calc_mu_max(), calc_re(), calc_rg(), calc_rh(), calc_slab_dipole(), DipolarP3MImpl< FloatType, Architecture >::calc_surface_term(), Galilei::calc_system_cms_position(), Galilei::calc_system_cms_velocity(), calc_total_charge(), CoulombTuningAlgorithm< FloatType, Architecture >::calculate_accuracy(), center_of_mass(), Coulomb::check_charge_neutrality(), ScriptInterface::Analysis::check_topology(), correct_position_shake(), correct_velocity_shake(), CoulombP3MImpl< FloatType, Architecture >::count_charged_particles(), DipolarP3MImpl< FloatType, Architecture >::count_magnetic_particles(), count_magnetic_particles(), dipolar_energy_correction(), dipolar_force_corrections(), distribute(), CoulombTuningAlgorithm< FloatType, Architecture >::fft_decomposition_veto(), FFTBackendLegacy< FloatType >::forward_fft(), CollisionDetection::gather_collision_queue(), BondBreakage::gather_global_queue(), gather_particle_data(), gather_traits_for_types(), get_ibm_particle_position(), get_particle_data(), get_particle_ids_parallel(), get_random_p_id(), gyration_tensor(), CollisionDetection::BindAtPointOfCollision::handle_collisions(), CollisionDetection::GlueToSurface::handle_collisions(), GpuParticleData::has_compatible_device(), FFTBackendLegacy< FloatType >::init(), Communication::init(), FFTBuffersLegacy< FloatType >::init_halo(), init_type_map(), System::System::integrate(), System::System::integrate_with_signal_handler(), is_valid_position(), ICCStar::iteration(), GatherParticleTraits< T >::join(), CoulombP3MImpl< FloatType, Architecture >::long_range_kernel(), DipolarP3MImpl< FloatType, Architecture >::long_range_kernel(), make_coulomb_scafacos(), make_dipolar_scafacos(), make_new_particle(), mindist(), modify_p3m_sums(), moment_of_inertia_matrix(), mpi_get_particles(), mpi_get_particles_local(), Observable_stat::mpi_reduce(), mpi_send_particle_data_local(), mpi_synchronize_max_seen_pid_local(), mpi_who_has_head(), mpi_who_has_local(), nbhood(), System::System::on_integration_start(), FFTBuffersLegacy< FloatType >::perform_scalar_halo_gather(), FFTBuffersLegacy< FloatType >::perform_scalar_halo_spread(), FFTBuffersLegacy< FloatType >::perform_vector_halo_gather(), FFTBuffersLegacy< FloatType >::perform_vector_halo_spread(), prefetch_particle_data(), propagate_vel_pos_sd(), rebuild_needed(), remove_particle(), ScriptInterface::System::rotate_system(), CellStructure::set_atom_decomposition(), CellStructure::set_hybrid_decomposition(), set_particle_pos(), CellStructure::set_regular_decomposition(), steepest_descent_step(), structure_factor(), synchronize_npt_state(), Constraints::ShapeBasedConstraint::total_force(), Constraints::ShapeBasedConstraint::total_normal_force(), CellStructure::update_ghosts_and_resort_particle(), System::System::update_used_propagations(), velocity_verlet_npt_finalize_p_inst(), velocity_verlet_npt_propagate_pos(), and System::System::veto_boxl_change().

communicator

Communicator communicator

extern

Definition at line 48 of file communication.cpp.

Referenced by ScriptInterface::CellSystem::CellSystem::CellSystem(), cuda_gather_gpus(), cuda_on_program_start(), ScriptInterface::walberla::LatticeWalberla::do_construct(), FFTBackendLegacy< FloatType >::init(), Communication::init(), System::System::maximal_cutoff(), CoulombP3M::sanity_checks_cell_structure(), DipolarP3M::sanity_checks_cell_structure(), CoulombP3M::sanity_checks_node_grid(), DipolarP3M::sanity_checks_node_grid(), and System::System::update_local_geo().

this_node

int this_node

extern

The number of this node.

Definition at line 69 of file communication.cpp.

Referenced by DipolarDirectSumGpu::add_long_range_forces(), CoulombP3MImpl< FloatType, Architecture >::add_long_range_forces_gpu(), benchmark_integration_step(), build_particle_node_parallel(), CoulombTuningAlgorithm< FloatType, Architecture >::calculate_accuracy(), System::System::calculate_forces(), Coulomb::check_charge_neutrality(), GpuParticleData::copy_forces_to_host(), cuda_gather_gpus(), cuda_on_program_start(), dipolar_energy_correction(), DipolarLayerCorrection::energy_correction(), get_particle_data(), get_particle_node_parallel(), ghost_communicator(), GpuParticleData::has_compatible_device(), is_poststorable(), is_prefetchable(), is_recv_op(), is_send_op(), DipolarDirectSumGpu::long_range_energy(), CoulombP3MImpl< FloatType, Architecture >::long_range_kernel(), DipolarP3MImpl< FloatType, Architecture >::long_range_kernel(), make_new_particle(), mpi_get_particles(), mpi_loop(), mpi_who_has_head(), remove_particle(), set_particle_pos(), CoulombTuningAlgorithm< FloatType, Architecture >::setup_logger(), DipolarTuningAlgorithm< FloatType, Architecture >::setup_logger(), velocity_verlet_npt_finalize_p_inst(), velocity_verlet_npt_propagate_pos(), and walberla_agrid_sanity_checks().