communication.hpp File Reference

This file contains the asynchronous MPI communication. More...

#include "MpiCallbacks.hpp"
#include <utils/Vector.hpp>
#include <boost/mpi/communicator.hpp>
#include <boost/mpi/environment.hpp>
#include <memory>
#include <utility>

Include dependency graph for core/communication.hpp:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	CommunicationEnvironment
struct	Communicator

Namespaces
namespace	Communication

Functions
MpiCallbacks &	Communication::mpiCallbacks ()
	Returns a reference to the global callback class instance.
void	mpi_loop ()
	Process requests from head node.

Variables
int	this_node
	The number of this node.
boost::mpi::communicator	comm_cart
	The communicator.
std::shared_ptr< KokkosHandle >	kokkos_handle
Communicator	communicator
std::unique_ptr< CommunicationEnvironment >	communication_environment

Detailed Description

This file contains the asynchronous MPI communication.

It is the header file for communication.cpp.

The asynchronous MPI communication is used during the script evaluation. Except for the head node that interprets the interface script, all other nodes wait in mpi_loop() for the head node to issue an action using MpiCallbacks::call(). mpi_loop() immediately executes an MPI_Bcast and therefore waits for the head node to broadcast a command, which is done by MpiCallbacks::call(). The request consists of a callback function with an arbitrary number of arguments.

To add new actions (e.g. to implement new interface functionality), do the following:

• write the mpi_* function that is executed on the head node
• write the mpi_*_local function that is executed on worker nodes
• register the local function with one of the REGISTER_CALLBACK macros

After this, your procedure is free to do anything. However, it has to be in (MPI) sync with what your new mpi_*_local does. This procedure is called immediately after the broadcast.

Definition in file core/communication.hpp.

Function Documentation

mpi_loop()

void mpi_loop

(

)

Process requests from head node.

Worker nodes main loop.

Definition at line 191 of file communication.cpp.

References Communication::MpiCallbacks::loop(), Communication::mpiCallbacks(), and this_node.

Variable Documentation

comm_cart

boost::mpi::communicator comm_cart

extern

The communicator.

Definition at line 68 of file communication.cpp.

Referenced by ComFixed::apply(), FFTBackendLegacy< FloatType >::backward_fft(), benchmark_integration_step(), DipolarP3MHeffte< FloatType, Architecture, FFTConfig >::calc_average_self_energy_k_space(), calc_mu_max(), calc_re(), calc_rg(), calc_rh(), calc_slab_dipole(), DipolarP3MHeffte< FloatType, Architecture, FFTConfig >::calc_surface_term(), Galilei::calc_system_cms_position(), Galilei::calc_system_cms_velocity(), calc_total_charge(), CoulombTuningAlgorithm< FloatType, Architecture, FFTConfig >::calculate_accuracy(), center_of_mass(), Coulomb::check_charge_neutrality(), ScriptInterface::Analysis::check_topology(), CommunicationEnvironment::CommunicationEnvironment(), correct_position_shake(), correct_velocity_shake(), CoulombP3MHeffte< FloatType, Architecture, FFTConfig >::count_charged_particles(), DipolarP3MHeffte< FloatType, Architecture, FFTConfig >::count_magnetic_particles(), count_magnetic_particles(), cuda_gather_gpus(), dipolar_energy_correction(), dipolar_force_corrections(), distribute(), CoulombTuningAlgorithm< FloatType, Architecture, FFTConfig >::fft_decomposition_veto(), FFTBackendLegacy< FloatType >::forward_fft(), gather_buffer_map(), CollisionDetection::gather_collision_queue(), BondBreakage::gather_global_queue(), gather_particle_data(), gather_traits_for_types(), get_ibm_particle_position(), get_particle_data(), get_particle_ids_parallel(), get_random_p_id(), gyration_tensor(), CollisionDetection::BindAtPointOfCollision::handle_collisions(), CollisionDetection::GlueToSurface::handle_collisions(), GpuParticleData::has_compatible_device(), FFTBackendLegacy< FloatType >::init(), CoulombP3MHeffte< FloatType, Architecture, FFTConfig >::init_cpu_kernels(), DipolarP3MHeffte< FloatType, Architecture, FFTConfig >::init_cpu_kernels(), FFTBuffersLegacy< FloatType >::init_halo(), init_type_map(), System::System::integrate(), System::System::integrate_with_signal_handler(), is_valid_position(), ICCStar::iteration(), GatherParticleTraits< T >::join(), CoulombP3MHeffte< FloatType, Architecture, FFTConfig >::long_range_kernel(), DipolarP3MHeffte< FloatType, Architecture, FFTConfig >::long_range_kernel(), make_coulomb_scafacos(), make_dipolar_scafacos(), make_new_particle(), mindist(), modify_p3m_sums(), moment_of_inertia_matrix(), mpi_get_particles(), mpi_get_particles_local(), Observable_stat::mpi_reduce(), mpi_send_particle_data_local(), mpi_synchronize_max_seen_pid_local(), mpi_who_has_head(), mpi_who_has_local(), nbhood(), System::System::npt_ensemble_init(), System::System::on_integration_start(), FFTBuffersLegacy< FloatType >::perform_scalar_halo_spread(), FFTBuffersLegacy< FloatType >::perform_vector_halo_gather(), FFTBuffersLegacy< FloatType >::perform_vector_halo_spread(), prefetch_particle_data(), propagate_vel_pos_sd(), rebuild_needed(), remove_particle(), ScriptInterface::System::rotate_system(), CoulombP3MHeffte< FloatType, Architecture, FFTConfig >::scaleby_box_l(), DipolarP3MHeffte< FloatType, Architecture, FFTConfig >::scaleby_box_l(), CellStructure::set_atom_decomposition(), CellStructure::set_hybrid_decomposition(), set_particle_pos(), CellStructure::set_regular_decomposition(), steepest_descent_step(), structure_factor(), System::System::synchronize_npt_state(), Constraints::ShapeBasedConstraint::total_force(), Constraints::ShapeBasedConstraint::total_normal_force(), CellStructure::update_ghosts_and_resort_particle(), System::System::update_used_propagations(), velocity_verlet_npt_finalize_p_inst(), velocity_verlet_npt_propagate_AVOVA_And(), velocity_verlet_npt_propagate_AVOVA_MTK(), velocity_verlet_npt_propagate_p_eps(), System::System::veto_boxl_change(), and vs_com_update_particles().

communication_environment

std::unique_ptr<CommunicationEnvironment> communication_environment

extern

Definition at line 70 of file communication.cpp.

Referenced by ScriptInterface::walberla::EKFFT::do_construct(), ScriptInterface::Coulomb::CoulombScafacos::do_construct(), ScriptInterface::Writer::H5md::do_construct(), ScriptInterface::Dipoles::DipolarScafacos::do_construct(), and ScriptInterface::walberla::EKFFTGPU::make_instance().

communicator

Communicator communicator

extern

Definition at line 69 of file communication.cpp.

Referenced by ScriptInterface::CellSystem::CellSystem::CellSystem(), CommunicationEnvironment::CommunicationEnvironment(), cuda_on_program_start(), ScriptInterface::walberla::LatticeWalberla::do_construct(), FFTBackendLegacy< FloatType >::init(), CoulombP3MHeffte< FloatType, Architecture, FFTConfig >::init_cpu_kernels(), DipolarP3MHeffte< FloatType, Architecture, FFTConfig >::init_cpu_kernels(), System::System::maximal_cutoff(), CoulombP3M::sanity_checks_cell_structure(), DipolarP3M::sanity_checks_cell_structure(), DipolarP3M::sanity_checks_node_grid(), and System::System::update_local_geo().

kokkos_handle

std::shared_ptr<KokkosHandle> kokkos_handle

extern

Definition at line 72 of file communication.cpp.

Referenced by CommunicationEnvironment::CommunicationEnvironment(), System::System::System(), and CommunicationEnvironment::~CommunicationEnvironment().

this_node

int this_node

extern

The number of this node.

Definition at line 74 of file communication.cpp.

Referenced by DipolarDirectSumGpu::add_long_range_forces(), CoulombP3MHeffte< FloatType, Architecture, FFTConfig >::add_long_range_forces_gpu(), benchmark_integration_step(), build_particle_node_parallel(), CoulombTuningAlgorithm< FloatType, Architecture, FFTConfig >::calculate_accuracy(), System::System::calculate_forces(), Coulomb::check_charge_neutrality(), GpuParticleData::copy_dip_fld_to_host(), GpuParticleData::copy_forces_to_host(), cuda_gather_gpus(), cuda_on_program_start(), dipolar_energy_correction(), DipolarLayerCorrection::energy_correction(), get_particle_data(), get_particle_node_parallel(), ghost_communicator(), GpuParticleData::has_compatible_device(), is_poststorable(), is_prefetchable(), is_recv_op(), is_send_op(), DipolarDirectSumGpu::long_range_energy(), CoulombP3MHeffte< FloatType, Architecture, FFTConfig >::long_range_kernel(), DipolarP3MHeffte< FloatType, Architecture, FFTConfig >::long_range_kernel(), make_new_particle(), mpi_get_particles(), mpi_loop(), mpi_who_has_head(), System::System::npt_ensemble_init(), prefetch_particle_data(), remove_particle(), set_particle_pos(), CoulombTuningAlgorithm< FloatType, Architecture, FFTConfig >::setup_logger(), DipolarTuningAlgorithm< FloatType, Architecture, FFTConfig >::setup_logger(), velocity_verlet_npt_finalize_p_inst(), velocity_verlet_npt_propagate_AVOVA_And(), velocity_verlet_npt_propagate_AVOVA_MTK(), velocity_verlet_npt_propagate_p_eps(), and walberla_agrid_sanity_checks().