+ Summary
+ Simulating multi-physics phenomena for real-world applications
+ states various challenges in scientific computing. Each individual
+ physical domain has behavior that is often described through a
+ distinct set of partial-differential equations that needs to be solved
+ in that domain. Their interaction is then achieved through
+ bidirectional exchange of suitable coupling data between all involved
+ domains. Partitioned coupling tackles multi-physics simulations by
+ glueing together separate models, typically implemented in separate
+ software environments. To facilitate such partitioned multi-physics
+ simulations effectively, so-called coupling libraries offer commonly
+ required functionality. We focus in particular on coupling through the
+ open-source library preCICE
+ (Chourdakis
+ et al., 2022), which offers functionality for data
+ communication, data mapping, coupling schemes, and more. In the most
+ basic setup, at least two executables call preCICE to perform a
+ coupled simulation. As additional software components, so-called
+ adapters bridge the gap between the preCICE API and the software
+ environments used by the coupled models. Creating and using this
+ overall setup for early development purposes is not only cumbersome,
+ but also very inefficient. The artificial solver testing environment
+ (ASTE) allows for replacing models coupled via preCICE with artificial
+ ones, potentially in parallel distributed across multiple ranks on
+ distributed memory. This helps in the development of preCICE,
+ adapters, or simulation setups by reducing the necessary software
+ components, simplifying execution workflows, and reducing runtime of
+ the case. In addition, ASTE provides performance and accuracy metrics
+ of the configured simulation setup.
+
+
+ Statement of need
+ [fig:dependency-graph]
+ illustrates the software stack required for a coupled simulation setup
+ using FEniCS and OpenFOAM as examples, and compares it to a simulation
+ setup using ASTE. Besides preCICE itself, core ingredients for
+ practical applications are preCICE API language bindings, preCICE
+ adapters, the simulation frameworks, and their dependencies. ASTE, on
+ the other hand, replaces coupled models and only requires a reduced
+ set of dependencies. It abstracts the computational complexity of the
+ models away by extracting the relevant information from VTK files
+ instead and passing extracted data to preCICE, potentially in parallel
+ on distributed memory. While the VTK files may stem from actual
+ simulations, ASTE can also generate artificial VTK files with
+ prescribed coupling data. On top of that, the entire tool chain of
+ ASTE enables easily altering the simulation setup through different
+ mesh partitionings, and specifically for the configuration of data
+ mappings in preCICE, ASTE can evaluate additional accuracy metrics of
+ used mappings.
+
+ Dependency graph between models, applications, and
+ libraries for a coupled simulation with FEniCS and OpenFOAM compared
+ to a dependency graph using
+ ASTE.
+ ![]()
+
+ From an application standpoint, ASTE provides a reproducible
+ environment which enables sharing and rerunning of scenarios,
+ regardless of the availability of involved software components. This
+ capability is particularly useful for debugging issues reported by
+ users of preCICE, who can share their scenarios (e.g., through the
+ preCICE
+ forum) for developers to analyze, even when the involved
+ software is unavailable due to licensing terms or being
+ closed-source.
+ A further crucial argument for emulating models with ASTE is
+ computational efficiency. For coupled simulations, the main
+ computational load is typically carried by the models instead of the
+ coupling library. Hence, running the original models repeatedly for
+ development purposes of preCICE or adapter components is both
+ time-consuming and inefficient. This inefficiency not only complicates
+ software development, but also applies to parameter tuning for
+ real-world applications, where the execution of involved models might
+ become prohibitively expensive already due to the problem size.
+ In this regard, ASTE provides a lightweight and valuable tool. It
+ enables the efficient development of preCICE by testing new features
+ on real-world applications in an artificial solver-like setup, e.g.,
+ for developing new communication algorithms
+ (Lindner,
+ 2019;
+ Totounferoush
+ et al., 2021) or to develop new mapping methods, e.g.,
+ (Ariguib,
+ 2022;
+ Chourdakis
+ et al., 2022;
+ Martin,
+ 2022;
+ Schneider
+ et al., 2023). In fact, testing and developing preCICE was the
+ use case behind the first prototype of ASTE, which was developed as
+ part of Lindner
+ (2019).
+ Beyond the development in preCICE, ASTE also fosters the development
+ of new adapter codes to be coupled via preCICE, as it aids in
+ debugging and enhances the transparency of data flow. Moreover,
+ combining preCICE’s performance instrumentation with the ASTE’s
+ flexibility and insight, it enables finding appropriate settings for
+ specific scenarios, as effectively demonstrated in the large-scale
+ example in Lindner et al.
+ (2020).
+ Although coupling libraries like MUI offer their own testing and
+ benchmarking infrastructure, e.g.,
+ MUI’s
+ testing framework, many tools do not provide such testing
+ environments at all. Instead, evaluating, testing and benchmarking of
+ these libraries relies on hard-coded solutions tailored to individual
+ test setups, e.g., the benchmarking performed by Valcke et al.
+ (2022).
+ ASTE covers a comprehensive, flexible and resuable toolchain for
+ development, testing, and parameter tuning.
+
+
+ Functionality & Use
+ The central interface of ASTE is given through a VTK mesh file,
+ which contains information about the geometric shape of the model we
+ emulate. The VTK files can be generated from mesh generation tools
+ (e.g., GMSH
+ (Geuzaine
+ & Remacle, 2009)), included
+ Python
+ scripts, other simulation software, or directly reused from
+ a
+ completed
+ preCICE simulation. Given a VTK file, ASTE offers different
+ algorithms to repartition them (e.g., through METIS
+ (Karypis
+ & Kumar, 2009)) for parallel runs. Moreover, ASTE can
+ generate artificial data using pre- or user-defined functions on the
+ mesh and store them in the VTK file format. The core module of ASTE
+ then reads the VTK file and passes the data to preCICE, potentially in
+ every time step of the coupled simulation. Once the simulation is
+ finished, the generated data is stored in another VTK file and can be
+ compared against the original artificial data. Performance metrics are
+ accessible through the
+ preCICE
+ performance framework.
+ While the core module of ASTE is written in C, the pre- and
+ postprocessing scripts are implemented in Python. The core module
+ relies on VTK
+ (Schroeder
+ et al., 2006),
+ Boost,
+ and MPI for parallel execution. It provides a command line interface
+ for simple simulations and can be configured in JSON
+ (Lohmann,
+ 2023) for more complex scenarios.
+ ASTE is hosted on
+ GitHub
+ and releases are published using
+ GitHub
+ releases. The documentation is part of the
+ ASTE
+ repository and rendered on
+ the
+ preCICE website. In addition, a
+ tutorial
+ and
+ ready-to-use
+ examples are available. Building is handled via
+ CMake
+ and, as part of the preCICE distribution
+ (Chen
+ et al., 2024), ASTE can be used through a
+ Vagrant
+ box.
+
+