Session

DescriptionPorting and tuning the performance of scientific applications on heterogeneous and increasingly complex supercomputer architectures is currently a manual and arduous task. Therefore, benchmarks are crucial for evaluating and improving the performance of applications on these systems. Benchmarks can be used as a proxy of the execution behaviors of scientific applications in a controlled and simpler environment, which provides an opportunity to extrapolate performance gains across new hardware, algorithm changes, and software updates. This mini symposium brings together interdisciplinary communities, who are using supercomputers, to discuss the challenges and opportunities in tracking, analyzing, and tuning application performance through the use of reproducible benchmarks. Specifically, we will address these critical topics: defining benchmarks to accurately capture scientific application behaviors; broadening performance metrics beyond time to solution, to reflect the impact of optimizations; modernizing the process of running benchmarks and analyzing their performance on different hardware architectures; proposing a standard for the definition of new benchmarks to improve reproducibility; identifying opportunities for software/hardware co-design across scientific applications and domains; and how to encourage contributions from the community through open-source benchmark implementations.