Skip to content

python interface updates and balancing option

Michael Buehlmann requested to merge master into master

The first two commits are updates to the (new) python interface:

  • allow setting the mismatch behavior (redistribute, allowed, disallowed) for more fine-grained control
  • update dependencies (pybind11 to version 2.6.1)

The third commit is a function in GenericIO:rebalanceSourceRanks that I originally wrote upon a request by Patricia. It is useful if the ranks that wrote the file were very unbalanced, which can happen for some analysis output.

  • The function redistributes the storage-ranks such that each reading-rank has a similar amount of data to read.
  • The reading can be faster or slower, depending on the machine and data.
  • The function only has an effect if MPI is being used and MismatchBehavior::MismatchRedistribute is selected.
  • The function needs to be manually called after openAndReadHeader

In case we don't want the last commit in the main branch, I can re-submit the merge-request with the first two commits only. However, since the rebalancing function only has an effect if manually called, it will not break/change any existing behavior.

Edited by Michael Buehlmann

Merge request reports