pack_masked_data_3d

Authors

Eric Maisonnave (CERFACS) and Pascal Terray (LOCEAN/IPSL)

Latest revision

29/05/2024

Purpose

Reduce the size of a tridimensional NetCDF variable extracted from a NetCDF dataset by storing only unmasked data associated with a constant land-sea mask in an output NetCDF dataset. The associated scale factors of the 2-D grid can also be stored in packed form in the output NetCDF dataset at the user option.

The output NetCDF variable stored in packed form in the output NetCDF file can be unpacked with unpack_masked_data_3d.

Further Details

Usage

$ pack_masked_data_3d \
  -f=input_netcdf_file \
  -v=netcdf_variable \
  -m=input_mesh_mask_netcdf_file \
  -o=output_netcdf_file \
  -x=lon1,lon2                      (optional) \
  -y=lat1,lat2                      (optional) \
  -t=time1,time2                    (optional) \
  -g=grid_type                      (optional : n, t, u, v, w, f) \
  -3d                               (optional) \
  -scalfac                          (optional) \
  -bigfile                          (optional) \
  -hdf5                             (optional) \
  -tlimited                         (optional)

By default

-x=

the whole longitude domain associated with the netcdf_variable

-y=

the whole latitude domain associated with the netcdf_variable

-t=

the whole time period associated with the netcdf_variable

-g=

the grid_type is set to n which means that the 2-D grid-mesh associated with the input NetCDF variable is assumed to be regular or Gaussian

-3d

the packed data are defined as a twodimensional NetCDF variable. However, if -3d is activated, the packed data is defined as an tridimensional NetCDF variable but with one dummy dimension (e.g., with a length equal to 1 )

-scalfac

The scale factors NetCDF variables in the input_mesh_mask_netcdf_file are not packed and copied to the output NetCDF file. If the -scalfac argument is specified, the associated packed scale factors NetCDF variables are copied and a dummy mask NetCDF variable is also written in the output NetCDF file

-bigfile

a NetCDF classical format file is created. If -bigfile is activated, the output NetCDF file is a 64-bit offset format file

-hdf5

a NetCDF classical format file is created. If -hdf5 is activated, the output NetCDF file is a NetCDF-4/HDF5 format file

-tlimited

the time dimension is defined as unlimited in the output NetCDF file. However, if -tlimited is activated, the time dimension is defined as limited in the output NetCDF file

Remarks

1. The -v=netcdf_variable argument specifies the NetCDF variable which must be packed and the -f=input_netcdf_file argument specifies that this NetCDF variable must be extracted from the NetCDF file input_netcdf_file.

2. The argument -m=input_mesh_mask_netcdf_file specifies that the land-sea mask to apply to the netcdf_variable must be read from the input_mesh_mask_netcdf_file.

   This input_mesh_mask_netcdf_file may be created by comp_clim_3d if the 2-D grid-mesh is regular or gaussian.

3. If the -x=lon1,lon2 and -y=lat1,lat2 arguments are missing, the geographical domain used in the packing operation is determined from the attributes of the input mesh-mask NetCDF variable named grid_typemask (e.g., the lon1_Eastern_limit, lon2_Western_limit, lat1_Southern_limit and lat2_Northern_limit attributes), which is read from the input NetCDF file input_mesh_mask_netcdf_file. If these attributes are missing, the whole geographical domain associated with the netcdf_variable is used to construct the packed NetCDF variable in the output dataset.

   The longitude or latitude range must be a vector of two integers specifying the first and last selected indices along each dimension. The indices are relative to 1 . Negative values are not allowed for lon1.

   Refer to comp_mask_3d for transforming geographical coordinates as indices or generating an appropriate mesh-mask before using pack_masked_data_3d.

4. If the -t=time1,time2 argument is missing the whole time period associated with the netcdf_variable is used to construct the packed NetCDF variable.

   The selected time period is a vector of two integers specifying the first and last time observations. The indices are relative to 1. Note that the output NetCDF file will have ntime = time2 - time1 + 1 time observations.

5. If -g= is set to t, u, v, w or f it is assumed that the NetCDF variable is from an experiment with the ORCA or NEMO model.

   If -g= is set to n, it is assumed that the 2-D grid-mesh is regular or Gaussian.

   The -g= argument is also used to determine the name of the NetCDF variable which contains the 2-D mesh-mask in the input_mesh_mask_netcdf_file (e.g., this variable is named grid_typemask).

6. The geographical shapes of the netcdf_variable (in the input_netcdf_file dataset) and the land-sea mask (in the input_mesh_mask_netcdf_file dataset) must agree.

7. The -3d argument specifies that the packed data must be stored as a tridimensional NetCDF variable with one dummy dimension in the output NetCDF file. By default, the packed data are stored as a twodimensional NetCDF variable.

8. The -scalfac argument specifies that the scale factors from the input_mesh_mask_netcdf_file must be packed and stored in the output NetCDF file if they exist. If the -scalfac argument is specified, a dummy mask is also written in the output NetCDF file. This allows further processing of the packed data by other NCSTAT procedures. Note that the grid_type of the packed mask and scale factors in the output NetCDF file is set to n.

   By default, the packed scale factors are not stored.

9. The -bigfile argument is allowed only if the NCSTAT software has been compiled with the _USE_NETCDF36 or _USE_NETCDF4 CPP macros (e.g., -D_USE_NETCDF36 or -D_USE_NETCDF4) and linked to the NetCDF 3.6 library or higher.

   If this argument is specified, the output_netcdf_file will be a 64-bit offset format file instead of a NetCDF classic format file. However, this argument is recognized in the procedure only if the NCSTAT software has been built with the _USE_NETCDF36 or _USE_NETCDF4 CPP macros.

10. The -hdf5 argument is allowed only if the NCSTAT software has been compiled with the _USE_NETCDF4 CPP macro (e.g., -D_USE_NETCDF4) and linked to the NetCDF 4 library or higher.

    If this argument is specified, the output_netcdf_file will be a NetCDF-4/HDF5 format file instead of a NetCDF classic format file. However, this argument is recognized in the procedure only if the NCSTAT software has been built with the _USE_NETCDF4 CPP macro.

11. Duplicate parameters are allowed, but this is always the last occurrence of a parameter which will be used for the computations. Moreover, the number of specified parameters must not be greater than the total number of allowed parameters.

Examples

1. For packing a NetCDF variable sosstsst extracted from the NetCDF dataset ST7_1m_0101_20012_grid_T_sosstsst.nc with the help of a land-sea mask extracted from the NetCDF file mesh_ocean.nc and store the results in a NetCDF file named packed_ST7_1m_0101_20012_grid_T_sosstsst.nc use the following command (note that the variable sosstsst is from a NEMO simulation since -g=t is specified) :

   $ pack_masked_data_3d \
     -f=ST7_1m_0101_20012_grid_T_sosstsst.nc \
     -v=sosstsst \
     -m=mesh_ocean.nc \
     -o=packed_ST7_1m_0101_20012_grid_T_sosstsst.nc \
     -g=t