NetCDF4 vs. HDF5 for large random access datasets

NetCDF4 is a layer on top of HDF5. NetCDF4 reads/writes specially structured HDF5 files. Performance of HDF5 and NetCDF4 is highly similar including on supercomputers. The idea behind NetCDF4 is a simpler API than HDF5. However, on Python h5py makes HDF5 read/write very easy. NetCDF4 is also easy to use from Python, so all things being nearly equal, perhaps the choice between NetCDF4 and HDF5 will be dictated by team consensus.

Using HDF5 from low-level languages is a little elaborate as compared to NetCDF4 ease of use. Using HDF or NetCDF4 from Python is far easier; these examples collapse down to a couple lines of code.

Leave a Comment