Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • G GenericIO
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 4
    • Issues 4
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 1
    • Merge requests 1
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • hacc
  • GenericIO
  • Merge requests
  • !2

python interface updates and balancing option

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Michael Buehlmann requested to merge master into master Dec 21, 2020
  • Overview 0
  • Commits 3
  • Pipelines 0
  • Changes 6

The first two commits are updates to the (new) python interface:

  • allow setting the mismatch behavior (redistribute, allowed, disallowed) for more fine-grained control
  • update dependencies (pybind11 to version 2.6.1)

The third commit is a function in GenericIO:rebalanceSourceRanks that I originally wrote upon a request by Patricia. It is useful if the ranks that wrote the file were very unbalanced, which can happen for some analysis output.

  • The function redistributes the storage-ranks such that each reading-rank has a similar amount of data to read.
  • The reading can be faster or slower, depending on the machine and data.
  • The function only has an effect if MPI is being used and MismatchBehavior::MismatchRedistribute is selected.
  • The function needs to be manually called after openAndReadHeader

In case we don't want the last commit in the main branch, I can re-submit the merge-request with the first two commits only. However, since the rebalancing function only has an effect if manually called, it will not break/change any existing behavior.

Edited Dec 21, 2020 by Michael Buehlmann
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: master

Virtual Help Desk | CELS Gitlab Instructions | Privacy Policy