Release History¶

Release: 2.3

Date: May 05, 2026

This version provides bug corrections, major improvements of OpenMP parallel reading in NCSTAT operators and improved documentation for many operators included in NCSTAT. An original moving block bootstrap approach has also been implemented for selecting the number of modes and testing the robustness of results of multivariate analysis procedures included in NCSTAT (e.g., NCSTAT operators comp_eof_3d, comp_eof_4d and comp_svd_3d).

This release requires version 2.3 or above of the STATPACK library (but version 2.4 is highly recommended for getting further speed improvements in NCSTAT multivariate procedures).

It is highly recommended that all users of NCSTAT upgrade to version 2.3 of NCSTAT.

Changes and updates:

For all NCSTAT operators: Bug corrections and better processing of the time coordinate variable in output NetCDF files produced by NCSTAT operators. In this version of NCSTAT, the time coordinate variable is always written as a double precision variable in output NetCDF files in order to avoid overflows or accuracy problems with the time coordinate variable.

For all NCSTAT operators: Errors messages for the command line arguments of all the NCSTAT operators have been substantially improved and it is now much easier for users to diagnose problems with his/her command line arguments of NCSTAT operators.

For all NCSTAT operators: It is now possible to create CDF5 format output files (available from the NetCDF 4.4 library or higher) by using the -cdf5 command line argument. Furthermore, output NetCDF-4/HDF5 format files, which are created when the -hdf5 command line argument is used, are now NetCDF-4/HDF5 classical model format files instead of NetCDF-4/HDF5 format files.

For all NCSTAT operators in which OpenMP parallel reading of NetCDF files can be used (e.g., when the CPP macro _PARALLEL_READ is used at compilation of NCSTAT), OpenMP parallel reading is used only if the input NetCDF files are classic, 64-bit offset or CDF5 format files. On the other hand, OpenMP parallel reading is desactivated for NetCDF-4/HDF5 files as it does not work for recent versions of the NetCDF library.

comp_clim_3d, comp_clim_4d, comp_clim_miss_3d, comp_clim_miss_4d, comp_stat_3d, comp_stat_4d, comp_stat_miss_3d:

the output mesh-mask NetCDF file, which can be created by these NCSTAT operators, now always includes geographical coordinate variables if these geographical coordinate variables are present in the input NetCDF file of the operator.

comp_mask_3d, comp_mask_4d:

the output mesh-mask NetCDF file, which can be created by these NCSTAT operators, now always includes geographical coordinate variables if these geographical coordinate variables are present and unidimensional in the input NetCDF file of the operator specified with the -f= command line argument.

comp_eof_3d, comp_eof_4d, comp_eof_miss_3d, comp_svd_3d, comp_svd2_3d:

Significant speed improvements, thanks to the use of version 2.4 of the STATPACK library, for computing the (full or partial) SVD of a real matrix for all algorithms specified with the -alg= argument, especially when OpenMP is used.

At the user option, a moving block bootstrap approach can now be used to obtain confidence intervals for the explained variances by the Principal Component (PC) or Singular Variable (SV) time series and to test the stability of the computed leading EOF and SVD modes. This bootstrap algorithm is computer intensive, but it can now be used for many datasets thanks to the use of the very fast randomized SVD algorithms available in versions 2.3 or 2.4 of the STATPACK library.

The “old” moving block bootstrap algorithm for computing confidence levels of SVD statistics used in the comp_svd_3d NCSTAT operator available in previous versions of NCSTAT, is still available, but in the (new) comp_svd2_3d NCSTAT operator. In other words, the comp_svd_3d and comp_svd2_3d operators differ only in the way of testing the results of a SVD analysis, otherwise their results are similar.

The documentation of these operators has also been improved.

comp_ortho_rot_eof_3d, comp_ortho_rot_eof_4d, comp_filt_rot_eof_3d, comp_filt_rot_eof_4d, pack_masked_data_3d, pack_masked_data_4d:

The documentation of these operators has also been improved.

Release: 2.2

Date: Jun 05, 2024

This version provides bug corrections, improved documentation for all operators and large speed improvements for multivariate analysis procedures included in NCSTAT.

New procedures for rotating partial Empirical Orthogonal Function (EOF), e.g., Principal Component Analysis (PCA), models in time or space states for improved physical interpretability of these EOF or PCA models are also added.

This update also provides many new procedures for performing power and cross-power spectral analysis of real multi-channel time series, for computing variance estimates in a selected frequency band from real multi-channel time series and for testing if two estimated multi-channel power spectra are the same or have the same shape based on a large variety of nonparametric statistical tests.

This release requires version 2.2 or above of the STATPACK library.

Changes and updates:

comp_eof_3d, comp_eof_4d, comp_eof_miss_3d:

Significant speed improvements thanks to the use of version 2.2 or 2.3 of the STATPACK library for computing the SVD of a real matrix for all algorithms specified by the -alg= argument, especially when OpenMP is used.

A new argument (e.g., -ortho) has been added to these procedures, which allows the user to control or improve the orthogonality of the EOFs and associated PC time series, which are computed by the inverse iteration and deflation algorithms (e.g., when -alg=inviter or -alg=deflate are specified in the procedure).

The documentation of these opeators has also been improved.

comp_svd_3d:

Significant speed improvements, thanks to the use of version 2.2 or 2.3 of the STATPACK library, for computing the SVD of a real matrix for all algorithms specified by the -alg= argument, especially when OpenMP is used.

A new argument (e.g., -ortho) has been added to this procedure, which allows the user to control or improve the orthogonality of the singular vectors and associated SV time series, which are computed by the inverse iteration and deflation algorithms (e.g., when -alg=inviter or -alg=deflate are specified in the procedure).

A new argument (e.g., -ba=) has also been added to this procedure, which allows the user to control which algorithm is used to compute the partial SVD of the bootstrap versions of the covariance matrix during the bootstrap phase of the procedure, independently of the value of the -alg= argument. The default value of the -ba= argument is rsvd (e.g., a randomized SVD algorithm) and this provides large speed improvements in the bootstrap SVD computations.

The documentation of this operator has also been improved.

comp_ortho_rot_eof_3d, comp_ortho_rot_eof_4d:

These 2 new NCSTAT operators perform an orthogonal rotation of a (partial) EOF or PCA model (e.g., a factor loading matrix and the associated standardized amplitude (PC) time series) computed from a previous EOF or PCA analysis of a tri- or fourdimensional NetCDF variable and store the rotated (partial) EOF model in a NetCDF file. A large set of orthogonal rotation criteria are included (e.g., quartimax, varimax and equamax rotation methods, etc.) to improve the interpretability of the factor loading matrix of the selected EOF or PCA model.

comp_filt_rot_eof_3d, comp_filt_rot_eof_4d:

These 2 new NCSTAT operators perform an orthogonal rotation of selected standardized Principal Components (PC) time series extracted from a previous Empirical Orthogonal Function (EOF) analysis or Principal Component Analysis (PCA) of a tridimensional or fourdimensional NetCDF variable towards low-frequency, high-frequency or band-pass components at the user option. The associated factor loading matrix (e.g., the associated scaled EOF patterns corresponding to the selected rotated PC time series) are also rotated and the new rotated EOF (or PCA) model is stored in a NetCDF file.

comp_loess_rot_eof_3d, comp_loess_rot_eof_4d:

These 2 new NCSTAT operators perform an orthogonal rotation of selected standardized Principal Components (PC) time series extracted from a previous Empirical Orthogonal Function (EOF) analysis or Principal Component Analysis (PCA) of a tri- or fourdimensional NetCDF variable towards low-frequency or high-frequency components using a LOESS smoother. The associated factor loading matrix (e.g., the associated scaled EOF patterns corresponding to the selected rotated PC time series) are also rotated and the new rotated EOF (or PCA) model is stored in a NetCDF file.

comp_spectvar_1d, comp_spectvar_3d, comp_spectvar_4d:

These 3 new NCSTAT operators compute variance estimates in a selected frequency band for real (multi-channel) time series, using an FFT algorithm and windowed filtering, and store the variance estimates in an output NetCDF file. These executables can be used in conjunction with the NCSTAT executables comp_fftfilter_1d, comp_fftfilter_3d, comp_fftfilter_4d, which filter real (multi-channel) time series in a selected frequency band by windowed filtering and also NCSTAT executables, which perform a power spectrum analysis of real (multi-channel) time series (e.g., comp_spectrum_1d, comp_cross_spectrum_1d, comp_cross_spectrum_3d, comp_cross_spectrum2_3d; see below).

comp_cross_spectrum_1d, comp_cross_spectrum_3d, comp_cross_spectrum2_3d:

These 3 new NCSTAT operators perform spectrum and cross-spectral analysis of real (multi-channel) time series, including statistical testing of the Power Spectral Density (PSD) and Cross-Spectral Density (CSD) estimates.

comp_spectrum_ratio_1d, comp_spectrum_ratio_3d, comp_spectrum_diff_3d:

These 3 new NCSTAT operators test if two multi-channel power spectra computed from two real multi-channel time series are significantly different or have a different shape using various nonparametric statistical tests.

comp_composite_3d, comp_composite_4d, comp_composite_miss_3d:

The documentation of these procedures has been fully revised and significantly improved.

Release: 2.1

Date: Jun 08, 2021

This version provides bug corrections, improved documentation, large speed improvements and very fast randomized algorithms for multivariate analysis procedures included in NCSTAT, improved propagation of input NetCDF variable attributes in the output NetCDF files produced by NCSTAT executables, improved computations of scale factors in mesh-mask NetCDF files produced by NCSTAT executables, new procedures for windowed filtering of time series, new procedures for vertical integrations in the atmosphere (or ocean) using scale factors included in mesh-mask NetCDF files produced by NCSTAT executables or from NEMO ocean model and, finally, new procedures for manipulation of fourdimensional NetCDF variables with missing values.

This release requires version 2.1 or above of the STATPACK library.

Changes and updates:

comp_clim_3d, comp_clim_4d, comp_clim_miss_3d, comp_clim_miss_4d, comp_stat_3d, comp_stat_4d, comp_stat_miss_3d:

When computing a mesh-mask NetCDF file, the -yl= argument, which specifies the latitude limits of the domain in degrees, is now needed only if the grid of the input NetCDF variable is a (regular) gaussian grid, but the domain is not the whole globe. Scale factors are now correctly estimated without the -yl= argument for a regular grid, even if the domain is not the whole globe.

Useful attributes of the input NetCDF variable (like grid_type, code and table attributes) are now propagated in the output NetCDF climatology or statistics files produced by these executables.

The documentation of these executables has been improved.

comp_clim_4d, comp_clim_miss_4d, comp_stat_4d:

A bug in the computation of the third (vertical) scale factor (included in mesh-mask NetCDF files produced by these executables) has been corrected and the computation of this third (vertical) scale factor has been improved (specifically the first and last elements of the third scale factor vector are now computed in all cases) such that this third scale factor vector can now be used for computing vertical integrals from fourdimensional NetCDF variables with the new NCSTAT executables comp_integral_4d and comp_integral_miss_4d.

comp_norm_4d:

A bug has been corrected. The old version of this exectuable was trying to open a mesh-mask NetCDF file in all cases, even when such a mesh-mask NetCDF file was not needed for the transformation of the input NetCDF variable.

comp_norm_3d, comp_norm_4d, comp_norm_miss_3d, comp_norm_miss_4d:

The documentation and the efficiency of the code of these executables have been significantly improved.

Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF files produced by these executables.

The new NCSTAT executable comp_norm_miss_4d allows now to process fourdimensional NetCDF variables with misssing values.

comp_season_3d, comp_season_4d, comp_season_miss_3d, comp_season_miss_4d:

Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF files produced by these executables.

The new NCSTAT executable comp_season_miss_4d allows now to process fourdimensional NetCDF variables with misssing values.

comp_integral_4d, comp_integral_miss_4d:

These 2 new NCSTAT executables compute vertical integrals from fourdimensional NetCDF variables with or without missing values, thanks to the improved computation of the third (vertical) scale factor descrived above for the comp_clim_4d, comp_clim_miss_4d, comp_stat_4d NCSTAT executables.

comp_fftfilter_1d, comp_fftfilter_3d, comp_fftfilter_4d:

These 3 new NCSTAT executables filter real (multichannel) time series in a selected frequency band by windowed filtering and store the filtered (multichannel) time series in an output NetCDF file. These executables are well adapted for filtering short length time series.

comp_svd_3d:

Two bugs have been corrected. The first bug concerns the computation of the metric with the -d2=dist3 option and the second bug concerns the bootstrapped computations when the -alg=deflate option is used.

Large speed improvements in all the (partial) SVD algorithms and implementation of a new and very fast randomized partial SVD algorithm (this new algorithm is selected when the -alg=rsvd argument is used).

Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF SVD files produced by this executable.

The documentation has been substantially improved.

comp_eof_3d, comp_eof_4d, comp_eof_miss_3d:

Large speed improvements in all the (partial) SVD algorithms and implementation of a new and very fast randomized partial SVD algorithm (this new algorithm is selected when the -alg=rsvd argument is used).

Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF EOF files produced by these executables.

The documentation of these executables has been substantially improved.

comp_project_eof_3d, comp_project_eof_4d:

The documentation of these executables has been substantially improved.

Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF files produced by these executables.

comp_stl_1d, comp_stl_3d, comp_stl_4d, comp_trend_1d, comp_trend_3d, comp_trend_4d:

Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF STL and trend files produced by these executables.

Release: 2.0

Date: Dec 3, 2018

First open source release

Updated Feb 17, 2021 for bug corrections

Updated May 07, 2021 for bug corrections

Updated June 07, 2021 for bug corrections

Release: 1.0

Date: March 5, 2004