Jens Henrik Göbbert, Mathis Bode, Andreas Lintermann and Herwig Zilken
Extracting scientific insight from large simulations is of crucial importance for science. Scientific advances are made only once the data produced by the simulations is processed into a meaningful analysis. But as simulations increase in size for more detail, post-processing is becoming the bottleneck on the way to the desired scientific insight.
In situ techniques, like in situ visualization, are known to play an important part in tackling this problem. Sparing some supercomputing time to process, structure, reduce and visualize the data in real-time during the simulation offers several benefits. In particular, when data reduction becomes inevitable, only during the simulation all relevant data about the simulated fields and any embedded geometry is readily available at the highest resolution and fidelity for critical decision making.
But even though in situ visualization has been successfully deployed over more than two decades, its use still has not gone "mainstream". Scientists today often have to abstain from meaningful in situ visualizations of their simulation data.
From discussions with developers and users of HPC codes for different research fields at the Forschungszentrum Jülich and the RWTH Aachen University two main reasons can be named. First, the individual implementation-, optimization- and coupling-costs to integrate the needed functionality to each simulation code and setup can often not be justified. Second, the usage of in situ visualization requires too much training for scientists who's research work does not focus on visualization in the first place. As the first is mainly a concern mentioned by the developers and addressed in different way by multiple software packages nowadays, the second is in no way less important. Hence, we are concerned on simplifying the use of in situ visualization at the JSC for both, developers and users.
For the integration of in situ visualization into multiple large scale simulation codes a light-weighted, flexible and easy-to-use coupling library has been developed. It covers the complexity and numerous options of in situ visualization and lowers the barriers to integrate visualization techniques into an existing simulation code. It allows to use the in situ functionality of VisIt as well as ParaView without disturbing the well-established work-flows in simulation code development. As it takes into account the fact that developers of visualization and simulation codes focus on different research results, the coupling is reduced to data adapters only, which convert between the data models of the simulation and visualization codes. This way the time-intensive cooperation between developers of the simulation- and visualization codes can be reduced. This library has been successfully integrated into highly scalable simulation codes running at Jülich Supercomputing Centre beginning with CIAO and psOpen of the Institute for Combustion Technology written in Fortran90 and ZFS of the Institute of Aerodynamics Aachen written in C++, which are all members of the High-Q Club for codes scaling to the complete JUQUEEN.
At the same time the integration of in situ visualization into existing simulation codes is simplified, solutions have been developed to raise the user acceptance of in situ visualization in their daily routine. Besides other benefits the simple access to remote visualization desktops with predefined desktop profiles for different visualization scenarios limit the burden of client-software configuration and give users with no visualization experience the first results in less time. Especially for users of research fields other than visualization we consider this of major importance.