Release HistoryΒΆ

Release:2.2
Date:Jun 05, 2024

This version provides bug corrections, improved documentation for all operators and large speed improvements for multivariate analysis procedures included in NCSTAT.

New procedures for rotating partial Empirical Orthogonal Function (EOF), e.g., Principal Component Analysis (PCA), models in time or space states for improved physical interpretability of these EOF or PCA models are also added.

This update also provides many new procedures for performing power and cross-power spectral analysis of real multi-channel time series, for computing variance estimates in a selected frequency band from real multi-channel time series and for testing if two estimated multi-channel power spectra are the same or have the same shape based on a large variety of nonparametric statistical tests.

This release requires version 2.2 or above of the STATPACK library.

Changes and updates:

comp_eof_3d, comp_eof_4d, comp_eof_miss_3d:

  • Significant speed improvements thanks to the use of version 2.2 or 2.3 of the STATPACK library for computing the SVD of a real matrix for all algorithms specified by the -alg= argument, especially when OpenMP is used.
  • A new argument (e.g., -ortho) has been added to these procedures, which allows the user to control or improve the orthogonality of the EOFs and associated PC time series, which are computed by the inverse iteration and deflation algorithms (e.g., when -alg=inviter or -alg=deflate are specified in the procedure).
  • The documentation of these opeators has also been improved.

comp_svd_3d:

  • Significant speed improvements, thanks to the use of version 2.2 or 2.3 of the STATPACK library, for computing the SVD of a real matrix for all algorithms specified by the -alg= argument, especially when OpenMP is used.
  • A new argument (e.g., -ortho) has been added to this procedure, which allows the user to control or improve the orthogonality of the singular vectors and associated SV time series, which are computed by the inverse iteration and deflation algorithms (e.g., when -alg=inviter or -alg=deflate are specified in the procedure).
  • A new argument (e.g., -ba=) has also been added to this procedure, which allows the user to control which algorithm is used to compute the partial SVD of the bootstrap versions of the covariance matrix during the bootstrap phase of the procedure, independently of the value of the -alg= argument. The default value of the -ba= argument is rsvd (e.g., a randomized SVD algorithm) and this provides large speed improvements in the bootstrap SVD computations.
  • The documentation of this operator has also been improved.

comp_ortho_rot_eof_3d, comp_ortho_rot_eof_4d:

  • These 2 new NCSTAT operators perform an orthogonal rotation of a (partial) EOF or PCA model (e.g., a factor loading matrix and the associated standardized amplitude (PC) time series) computed from a previous EOF or PCA analysis of a tri- or fourdimensional NetCDF variable and store the rotated (partial) EOF model in a NetCDF file. A large set of orthogonal rotation criteria are included (e.g., quartimax, varimax and equamax rotation methods, etc.) to improve the interpretability of the factor loading matrix of the selected EOF or PCA model.

comp_filt_rot_eof_3d, comp_filt_rot_eof_4d:

  • These 2 new NCSTAT operators perform an orthogonal rotation of selected standardized Principal Components (PC) time series extracted from a previous Empirical Orthogonal Function (EOF) analysis or Principal Component Analysis (PCA) of a tridimensional or fourdimensional NetCDF variable towards low-frequency, high-frequency or band-pass components at the user option. The associated factor loading matrix (e.g., the associated scaled EOF patterns corresponding to the selected rotated PC time series) are also rotated and the new rotated EOF (or PCA) model is stored in a NetCDF file.

comp_loess_rot_eof_3d, comp_loess_rot_eof_4d:

  • These 2 new NCSTAT operators perform an orthogonal rotation of selected standardized Principal Components (PC) time series extracted from a previous Empirical Orthogonal Function (EOF) analysis or Principal Component Analysis (PCA) of a tri- or fourdimensional NetCDF variable towards low-frequency or high-frequency components using a LOESS smoother. The associated factor loading matrix (e.g., the associated scaled EOF patterns corresponding to the selected rotated PC time series) are also rotated and the new rotated EOF (or PCA) model is stored in a NetCDF file.

comp_spectvar_1d, comp_spectvar_3d, comp_spectvar_4d:

  • These 3 new NCSTAT operators compute variance estimates in a selected frequency band for real (multi-channel) time series, using an FFT algorithm and windowed filtering, and store the variance estimates in an output NetCDF file. These executables can be used in conjunction with the NCSTAT executables comp_fftfilter_1d, comp_fftfilter_3d, comp_fftfilter_4d, which filter real (multi-channel) time series in a selected frequency band by windowed filtering and also NCSTAT executables, which perform a power spectrum analysis of real (multi-channel) time series (e.g., comp_spectrum_1d, comp_cross_spectrum_1d, comp_cross_spectrum_3d, comp_cross_spectrum2_3d; see below).

comp_cross_spectrum_1d, comp_cross_spectrum_3d, comp_cross_spectrum2_3d:

  • These 3 new NCSTAT operators perform spectrum and cross-spectral analysis of real (multi-channel) time series, including statistical testing of the Power Spectral Density (PSD) and Cross-Spectral Density (CSD) estimates.

comp_spectrum_ratio_1d, comp_spectrum_ratio_3d, comp_spectrum_diff_3d:

  • These 3 new NCSTAT operators test if two multi-channel power spectra computed from two real multi-channel time series are significantly different or have a different shape using various nonparametric statistical tests.

comp_composite_3d, comp_composite_4d, comp_composite_miss_3d:

  • The documentation of these procedures has been fully revised and significantly improved.
Release:2.1
Date:Jun 08, 2021

This version provides bug corrections, improved documentation, large speed improvements and very fast randomized algorithms for multivariate analysis procedures included in NCSTAT, improved propagation of input NetCDF variable attributes in the output NetCDF files produced by NCSTAT executables, improved computations of scale factors in mesh-mask NetCDF files produced by NCSTAT executables, new procedures for windowed filtering of time series, new procedures for vertical integrations in the atmosphere (or ocean) using scale factors included in mesh-mask NetCDF files produced by NCSTAT executables or from NEMO ocean model and, finally, new procedures for manipulation of fourdimensional NetCDF variables with missing values.

This release requires version 2.1 or above of the STATPACK library.

Changes and updates:

comp_clim_3d, comp_clim_4d, comp_clim_miss_3d, comp_clim_miss_4d, comp_stat_3d, comp_stat_4d, comp_stat_miss_3d:

  • When computing a mesh-mask NetCDF file, the -yl= argument, which specifies the latitude limits of the domain in degrees, is now needed only if the grid of the input NetCDF variable is a (regular) gaussian grid, but the domain is not the whole globe. Scale factors are now correctly estimated without the -yl= argument for a regular grid, even if the domain is not the whole globe.
  • Useful attributes of the input NetCDF variable (like grid_type, code and table attributes) are now propagated in the output NetCDF climatology or statistics files produced by these executables.
  • The documentation of these executables has been improved.

comp_clim_4d, comp_clim_miss_4d, comp_stat_4d:

  • A bug in the computation of the third (vertical) scale factor (included in mesh-mask NetCDF files produced by these executables) has been corrected and the computation of this third (vertical) scale factor has been improved (specifically the first and last elements of the third scale factor vector are now computed in all cases) such that this third scale factor vector can now be used for computing vertical integrals from fourdimensional NetCDF variables with the new NCSTAT executables comp_integral_4d and comp_integral_miss_4d.

comp_norm_4d:

  • A bug has been corrected. The old version of this exectuable was trying to open a mesh-mask NetCDF file in all cases, even when such a mesh-mask NetCDF file was not needed for the transformation of the input NetCDF variable.

comp_norm_3d, comp_norm_4d, comp_norm_miss_3d, comp_norm_miss_4d:

  • The documentation and the efficiency of the code of these executables have been significantly improved.
  • Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF files produced by these executables.
  • The new NCSTAT executable comp_norm_miss_4d allows now to process fourdimensional NetCDF variables with misssing values.

comp_season_3d, comp_season_4d, comp_season_miss_3d, comp_season_miss_4d:

  • Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF files produced by these executables.
  • The new NCSTAT executable comp_season_miss_4d allows now to process fourdimensional NetCDF variables with misssing values.

comp_integral_4d, comp_integral_miss_4d:

  • These 2 new NCSTAT executables compute vertical integrals from fourdimensional NetCDF variables with or without missing values, thanks to the improved computation of the third (vertical) scale factor descrived above for the comp_clim_4d, comp_clim_miss_4d, comp_stat_4d NCSTAT executables.

comp_fftfilter_1d, comp_fftfilter_3d, comp_fftfilter_4d:

  • These 3 new NCSTAT executables filter real (multichannel) time series in a selected frequency band by windowed filtering and store the filtered (multichannel) time series in an output NetCDF file. These executables are well adapted for filtering short length time series.

comp_svd_3d:

  • Two bugs have been corrected. The first bug concerns the computation of the metric with the -d2=dist3 option and the second bug concerns the bootstrapped computations when the -alg=deflate option is used.
  • Large speed improvements in all the (partial) SVD algorithms and implementation of a new and very fast randomized partial SVD algorithm (this new algorithm is selected when the -alg=rsvd argument is used).
  • Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF SVD files produced by this executable.
  • The documentation has been substantially improved.

comp_eof_3d, comp_eof_4d, comp_eof_miss_3d:

  • Large speed improvements in all the (partial) SVD algorithms and implementation of a new and very fast randomized partial SVD algorithm (this new algorithm is selected when the -alg=rsvd argument is used).
  • Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF EOF files produced by these executables.
  • The documentation of these executables has been substantially improved.

comp_project_eof_3d, comp_project_eof_4d:

  • The documentation of these executables has been substantially improved.
  • Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF files produced by these executables.

comp_stl_1d, comp_stl_3d, comp_stl_4d, comp_trend_1d, comp_trend_3d, comp_trend_4d:

  • Useful attributes of the input NetCDF variable (like units, grid_type, code and table attributes) are now propagated in the output NetCDF STL and trend files produced by these executables.
Release:2.0
Date:Dec 3, 2018

First open source release

Updated Feb 17, 2021 for bug corrections

Updated May 07, 2021 for bug corrections

Updated June 07, 2021 for bug corrections

Release:1.0
Date:March 5, 2004
Flag Counter