Parallel I/O Benchmarks, Applications, Traces

Below is a list of parallel I/O benchmarks, applications, and traces I am aware of (in no particular order). Let me (Rajeev Thakur) know if you have any additions or corrections to the list.

Updated: July 18, 2023

Benchmarks

  1. H5bench: A Parallel I/O Benchmark Suite for HDF-5
    https://github.com/hpc-io/h5bench

  2. DLIO Benchmark for Scientific Deep Learning Applications
    https://github.com/hariharan-devarajan/dlio_benchmark

  3. IOR and mdtest
    https://github.com/hpc/ior

  4. WRFIO
    https://bitbucket.org/pmalakar/wrfio

  5. MACSio Proxy App
    https://codesign.llnl.gov/macsio.php

  6. PnetCDF Benchmarks
    http://cucis.ece.northwestern.edu/projects/PnetCDF/#Benchmarks

  7. Performance Test in Parallel I/O (PIO) Library
    http://ncar.github.io/ParallelIO/ (see tests/performance directory)

  8. HACC I/O
    https://asc.llnl.gov/CORAL-benchmarks/#hacc

  9. FLASH I/O
    http://www.ucolick.org/~zingale/flash_benchmark_io/
    An updated version for newer versions of parallel NetCDF and HDF-5 is available     here.

  10. Effective I/O Bandwidth Benchmark (b_eff_io)
    http://www.hlrs.de/mpi/b_eff_io/

  11. NAS BTIO Benchmark
    http://www.nas.nasa.gov/Resources/Software/npb.html (see BT in NPB3.3-MPI)

  12. MPI Tile I/O
    http://www.mcs.anl.gov/pio-benchmark/

  13. FS_TEST
    https://github.com/fs-test/fs_test (see this as a README)

  14. Trace-Driven Scientific I/O Benchmarks
    http://www.cs.rochester.edu/research/os/hec-io/

  15. HPIO and S3asim
    http://www.eecs.northwestern.edu/~aching/research_webpage/io.html

  16. Noncontig
    http://www.mcs.anl.gov/pio-benchmark/

  17. ioperf test in Silo
    https://github.com/LLNL/Silo

  18. Pattern I/O Benchmark (John Shalf, LBNL)
    pattern_io.tar.gz

  19. MADbench2
    http://crd.lbl.gov/departments/computational-science/computational-cosmology-center/c3-research/madbench2/

  20. QIO
    http://usqcd.jlab.org/usqcd-docs/qio/

  21. PIO-Bench
    ftp://ftp.parl.clemson.edu/pub/pio-bench
    An article describing it is at     http://www.clustermonkey.net/content/view/87/32/.

Applications

  1. Earth System Modeling Framework (ESMF)
    http://www.earthsystemmodeling.org/

  2. Nek5000
    https://nek5000.mcs.anl.gov/

  3. NekCEM
    https://nekcem.mcs.anl.gov/index.php/Main_Page

  4. PISM: Parallel Ice Sheet Model
    http://www.pism-docs.org

  5. MOAB
    https://press3.mcs.anl.gov/sigma/moab-library/

  6. PFLOTRAN
    http://www.pflotran.org/

  7. Cactus
    http://www.cactuscode.org/

  8. mpiBLAST
    http://www.mpiblast.org/

  9. FLASH
    http://www.flash.uchicago.edu/
    Also see FLASH I/O Benchmark above.

  10. Community Earth System Model (CESM)
    http://www.cesm.ucar.edu/

  11. Regional Ocean Modeling System (ROMS)
    https://www.myroms.org/

  12. Weather Research & Forecasting Model (WRF)
    http://www2.mmm.ucar.edu/wrf/users/download/get_source.html

  13. CM1
    http://www2.mmm.ucar.edu/people/bryan/cm1/

  14. Qbox
    http://qboxcode.org/

  15. GADGET
    http://www.mpa-garching.mpg.de/gadget/

  16. MILC
    http://www.physics.utah.edu/~detar/milc/

  17. MPQC
    http://www.mpqc.org/

  18. NaSt3DGP
    http://wissrech.ins.uni-bonn.de/research/projects/NaSt3DGP/index.htm

  19. SIESTA
    http://www.icmab.es/siesta/

  20. CP2K
    http://www.cp2k.org/

  21. LAMMPS
    http://lammps.sandia.gov/

Traces

  1. HPC Application I/O Traces
    http://library.ucsd.edu/dc/object/bb95276921

  2. Darshan
    http://www.mcs.anl.gov/research/projects/darshan/data/

  3. Sandia
    http://www.cs.sandia.gov/Scalable_IO/SNL_Trace_Data/index.html

  4. MAMBO
    http://www.cs.umd.edu/projects/hpsl/mambo/index.html

  5. IO-SIG Software for I/O Signature
    http://www.cs.iit.edu/~scs/assets/projects/iosig/index.html