Containers in HPC: a survey

OS-level virtualization (containers) has become a popular alternative to hypervisor-based virtualization. From a system-administration point-of-view, containers enable support for user-defined software stacks, thus freeing users of restrictions imposed by the host’s pre-configured software environment. In high performance computing (HPC), containers inspire special interest due to their potentially low overheads on performance. Moreover, they also bring benefits in portability and scientific reproducibility. Despite the potential advantages, the adoption of containers in HPC has been relatively slow, mainly due to specific requirements of the field. These requirements gave rise to various HPC-focused container implementations. Besides unprivileged container execution, they offer different degrees of automation of system-specific optimizations, which are necessary for optimal performance. When we looked into the scientific literature on containers applied to HPC, we were unable to find an up-to-date overview of the state-of-the-art. For this reason, we developed this extensive survey, including 93 carefully selected works. Overall, based on our survey, we argue that issues related to performance overhead are mostly solved. There is, however, a clear trade-off between performance and portability, since optimal performance often depends on host-specific optimizations. A few works propose solutions to mitigate this issue, but there is still room for improvement. Besides, we found surprisingly few works that deal with portability between dedicated HPC systems and public cloud platforms.