Challenge
E-CAM is interested in the challenge of bridging up timescales. To study molecular dynamics with atomistic detail, timesteps must be used on the order of a femto-second. Many problems in biological chemistry, materials science, and other ends involve events that only spontaneously occur after a millisecond or longer (for example, biomolecular conformational changes, or nucleation processes). That means that around 1012 time steps would be needed to see a single millisecond-scale event. This is the problem of “rare events” in theoretical and computational chemistry. Modern supercomputers are beginning to make it possible to obtain trajectories long enough to observe some of these processes, but to fully characterize a transition with proper statistics, many examples are needed. And in order to obtain many examples, the same application must be run thousands of times with varying inputs. To manage this kind of computation, a task scheduling library is needed
Solution and benefits
The development of a python library, in collaboration with PRACE. This library builds on top of the scalable analytics framework Dask and enables it to resiliently manage multi-node and multiarchitecture environments. This offers exciting possibilities in the areas of interactive supercomputing and burst supercomputing. A white paper focused on the library was written in collaboration with PRACE and is available here.
The main elements of the mentioned scheduling library are: task de definition, a task scheduling (handled in Python) and task execution (facilitated by the MPI layer). While traditionally an HTC workload is looked down upon in the HPC space, the scientific use case for extreme-scale resources exists and algorithms that require a coordinated approach make efficient libraries that implement this approach increasingly important in the HPC space. The 5 Peta op booster technology of JURECA is an interesting concept with respect to this approach since the offloading approach of heavy computation marries perfectly to the concept outlined here.
Reference
Alan O’Cais, David Swenson, Mariusz Uchronski, & Adam Wlodarczyk. (2019, August 14). Task Scheduling Library for Optimising Time-Scale Molecular Dynamics Simulations. Zenodo. http://doi.org/10.5281/zenodo.3527643
