TempestOnlineMapIO.cpp File Reference

#include "FiniteElementTools.h"
#include "moab/Remapping/TempestOnlineMap.hpp"
#include "moab/TupleList.hpp"
#include "netcdfcpp.h"

Include dependency graph for TempestOnlineMapIO.cpp:

Go to the source code of this file.

Enumerations
enum	MapFileFormat { MAP_FORMAT_UNKNOWN = 0 , MAP_FORMAT_CLASSIC = 1 , MAP_FORMAT_NETCDF4 = 2 }

Functions
void	print_progress (const int barWidth, const float progress, const char *message)
static int	detectMapFileNetCDFFormat (const char *path)
	Inspect the file header to classify NetCDF format. Reads only the first 8 bytes — relies on the well-known "CDF\\xNN" and HDF5 magic signatures. Returns MAP_FORMAT_UNKNOWN if the file cannot be opened or the signature does not match. Safe to call from a single rank. More...

Variables
static constexpr int	BUFFERED_READ_NNZ_THRESHOLD = 3000000
	NNZ threshold: maps with nS <= this value use the buffered read strategy. Maps with nS > this value use direct parallel I/O (if available). Default 3M entries corresponds to ~36 MB of raw data (row+col+S). More...
static constexpr int	BUFFERED_READ_CHUNK_BYTES = 64 * 1024
	Buffer size in bytes for each chunk read by rank 0 in buffered mode. Each sparse matrix entry is 12 bytes (2 ints + 1 double), so 64KB holds ~5461 entries. Larger buffers reduce the number of read+scatter rounds but increase peak memory on rank 0. More...

Enumeration Type Documentation

MapFileFormat

enum MapFileFormat

Enumerator
MAP_FORMAT_UNKNOWN
MAP_FORMAT_CLASSIC
MAP_FORMAT_NETCDF4

Definition at line 1306 of file TempestOnlineMapIO.cpp.

{
    MAP_FORMAT_UNKNOWN = 0,
    MAP_FORMAT_CLASSIC = 1,  // CDF-1, CDF-2, CDF-5
    MAP_FORMAT_NETCDF4 = 2   // NetCDF-4 / HDF5
};

Function Documentation

detectMapFileNetCDFFormat()

static int detectMapFileNetCDFFormat ( const char *  path )

static

Inspect the file header to classify NetCDF format. Reads only the first 8 bytes — relies on the well-known "CDF\\xNN" and HDF5 magic signatures. Returns MAP_FORMAT_UNKNOWN if the file cannot be opened or the signature does not match. Safe to call from a single rank.

References:

• NetCDF classic format spec: https://docs.unidata.ucar.edu/nug/current/file_format_specifications.html ("CDF\\x01" / "CDF\\x02" / "CDF\\x05")
• HDF5 superblock signature: \x89 H D F \r \n \x1a \n

Definition at line 1322 of file TempestOnlineMapIO.cpp.

{
    std::FILE* fp = std::fopen( path, "rb" );
    if( !fp ) return MAP_FORMAT_UNKNOWN;
 
    unsigned char magic[8] = { 0 };
    const size_t nread     = std::fread( magic, 1, sizeof( magic ), fp );
    std::fclose( fp );
 
    if( nread < 4 ) return MAP_FORMAT_UNKNOWN;
 
    // Classic NetCDF families: 'C','D','F' + version byte (0x01, 0x02, or 0x05)
    if( magic[0] == 'C' && magic[1] == 'D' && magic[2] == 'F' )
    {
        if( magic[3] == 0x01 || magic[3] == 0x02 || magic[3] == 0x05 ) return MAP_FORMAT_CLASSIC;
    }
 
    // HDF5 signature (used by NetCDF-4)
    if( nread >= 8 && magic[0] == 0x89 && magic[1] == 'H' && magic[2] == 'D' && magic[3] == 'F' && magic[4] == 0x0D &&
        magic[5] == 0x0A && magic[6] == 0x1A && magic[7] == 0x0A )
    {
        return MAP_FORMAT_NETCDF4;
    }
 
    return MAP_FORMAT_UNKNOWN;
}

References MAP_FORMAT_CLASSIC, MAP_FORMAT_NETCDF4, and MAP_FORMAT_UNKNOWN.

Referenced by moab::TempestOnlineMap::ReadParallelMap().

print_progress()

void print_progress	(	const int	barWidth,
		const float	progress,
		const char *	message
	)

Definition at line 1246 of file TempestOnlineMapIO.cpp.

{
    std::cout << message << " [";
    int pos = barWidth * progress;
    for( int i = 0; i < barWidth; ++i )
    {
        if( i < pos )
            std::cout << "=";
        else if( i == pos )
            std::cout << ">";
        else
            std::cout << " ";
    }
    std::cout << "] " << int( progress * 100.0 ) << " %\r";
    std::cout.flush();
}

Variable Documentation

BUFFERED_READ_CHUNK_BYTES

constexpr int BUFFERED_READ_CHUNK_BYTES = 64 * 1024

staticconstexpr

Buffer size in bytes for each chunk read by rank 0 in buffered mode. Each sparse matrix entry is 12 bytes (2 ints + 1 double), so 64KB holds ~5461 entries. Larger buffers reduce the number of read+scatter rounds but increase peak memory on rank 0.

Definition at line 1300 of file TempestOnlineMapIO.cpp.

Referenced by moab::TempestOnlineMap::ReadParallelMap().

BUFFERED_READ_NNZ_THRESHOLD

constexpr int BUFFERED_READ_NNZ_THRESHOLD = 3000000

staticconstexpr

NNZ threshold: maps with nS <= this value use the buffered read strategy. Maps with nS > this value use direct parallel I/O (if available). Default 3M entries corresponds to ~36 MB of raw data (row+col+S).

Definition at line 1294 of file TempestOnlineMapIO.cpp.

Referenced by moab::TempestOnlineMap::ReadParallelMap().