comp_fftfilter_1d

Authors

Pascal Terray (LOCEAN/IPSL)

Latest revision

28/05/2024

Purpose

Filter a real time series in a selected frequency band by windowed filtering [Iacobucci_Noullez] . The time series is extracted from a uni- or bidimensional variable readed from a NetCDF dataset and can also be detrended before windowed filtering at the user option.

The windowed filter is applied to the time series in the frequency domain [Iacobucci_Noullez] . The filter is obtained by convolving a raised-cosine window with the ideal rectangular filter response function. This filter is stationary and symmetric, therefore, it induces no phase-shift and is a good candidate for extracting frequency-defined series components from short-length time series.

Applying the filter in the frequency domain assumes implicitly that the time series is part of a periodic infinite series whose period is exactly equal to the length of the analyzed time series. The selected ideal filter gain vector (in the frequency domain) is convolved with the Fourier transform of a raised-cosine window to get the final filter frequency response. This filter frequency response is then convolved with the Fourier transform of the time series and the resulting sequence is inverted back in the time domain to obtain the final filtered time series. The specific form of the raised-cosine window can be controlled by the user with the help of the -win= argument, described below.

Additionally, the filtering can be done separately for different segments of equal length of the selected time series if this time series is not continuous in time (see the -p= argument).

This procedure returns the filtered real time series in a NetCDF dataset. If the NetCDF variable is trid or fourdimensional use comp_fftfilter_3d or comp_fftfilter_4d, respectively, instead of comp_fftfilter_1d. If the time series has a seasonal (or diurnal) cycle, use comp_stl_1d in order to estimate and remove the harmonic components of the time series before using comp_fftfilter_1d.

Finally, use comp_spectvar_1d to compute variance estimates in the selected frequency band for the real time series before or after filtering the real time series.

Further Details

Usage

$ comp_fftfilter_1d \
  -f=input_netcdf_file \
  -v=netcdf_variable \
  -t=time1,time2                           (optional) \
  -o=output_netcdf_file                    (optional) \
  -ni=index_for_2d_netcdf_variable         (optional) \
  -p=period_length                         (optional) \
  -pl=minimum_period                       (optional) \
  -ph=maximum_period                       (optional) \
  -tr=trend_removal                        (optional : 0, 1, 2, 3, -1, -2, -3) \
  -win=window_choice                       (optional : 0.5 > 1.) \
  -mi=missing_value                        (optional) \
  -double                                  (optional) \
  -hdf5                                    (optional) \
  -tlimited                                (optional)

By default

-t=
the whole time period associated with the netcdf_variable
-o=
the output_netcdf_file is named filt_netcdf_variable.nc
-ni=
if the netcdf_variable is bidimensional, the first time series is used
-p=
the period_length is set to time2 - time1 + 1 , which means that the time series is considered as continuous with only one time segment
-pl=
the minimum_period is set to 0, which means that no filtering is done for the shorter periods
-ph=
the maximum_period is set to 0, which means that no filtering is done for the longer periods
-tr=0
trend_removal is set to 0, which means that no detrending is done before filtering
-win=0.54
the Hamming window is convolved with the ideal filter response
-mi=
the missing_value for the output variable is equal to 1.e+20
-double
the results are stored as single-precision floating point numbers in the output NetCDF file. If -double is activated, the results are stored as double-precision floating point numbers
-hdf5
a NetCDF classical format file is created. If -hdf5 is activated, the output NetCDF file is a NetCDF-4/HDF5 format file
-tlimited
the time dimension is defined as unlimited in the output NetCDF file. However, if -tlimited is activated, the time dimension is defined as limited in the output NetCDF file

Remarks

  1. The -v=netcdf_variable argument specifies the NetCDF variable for which a windowed filtering operation must be computed and the -f=input_netcdf_file argument specifies that this NetCDF variable must be extracted from the NetCDF file input_netcdf_file.

  2. If the -t=time1,time2 argument is missing the whole time period associated with the netcdf_variable is used to decompose the time series.

    The selected time period is a vector of two integers specifying the first and last time observations. The indices are relative to 1. Note that the output NetCDF file will have ntime = time2 - time1 + 1 time observations.

  3. The -ni=index_for_2d_netcdf_variable argument specifies the index for selecting the time series if the netcdf_variable is a 2D NetCDF variable. By default, the first time series is used, which is equivalent to set index_for_2d_netcdf_variable to 1.

  4. If the -p= argument is specified, the filtering is applied separately for each time segment of length period_length (as determined by the value of the -p= argument). The whole selected time period (e.g., time2 - time1 + 1 ) must also be a multiple of the period_length.

  5. The -pl= argument specifies the minimum period of oscillation of the filtered time series. The minimum_period is expressed in number of time observations.

    Do not use the -pl= argument or use -pl=0 for high-pass filtering frequencies corresponding to periods shorter than -ph=PH .

    The -pl= argument is a positive integer equal to 0 or greater than 2.

  6. The -ph= argument specifies the maximum period of oscillation of the filtered time series. The maximum_period is expressed in number of time observations. Do not use the -ph= argument or use -ph=0 for low-pass filtering frequencies corresponding to periods longer than -pl=PL . For example, -pl=6 (or 18) and -ph=32 (or 96) select periods between 1.5 and 8 years for quarterly (monthly) time series.

    The -ph= argument is a positive integer equal to 0 or greater than 2 and less than the length of the time series or the period_length if the -p= argument is used.

    The -ph= argument must also be greater or equal to the -pl= argument if both are specified.

  7. Setting -pl= and -ph= to the same value P is allowed. In this case, an -ideal- band-pass filter with peak response near one at the single period P is computed and applied to the time series.

  8. Setting -pl=PL, -ph=PH and PH``<``PL is also allowed and performs band rejection of periods between PH and PL. In that case, the meaning of the -pl= and -ph= arguments reversed.

  9. Setting both -pl=0 and -ph=0 is also allowed. In that case, no frequencies filtering is done, but the data may be detrended if the -tr= argument is used with a value of 1, 2 or 3.

  10. The -tr= argument specifies pre- and post-filtering processing of the time series. If:

    • -tr=+/-1, the mean of the time series is removed before time filtering
    • -tr=+/-2, the drift from the time series is removed before time filtering. The drift for the time series is estimated using the formula : drift = ( tseries(ntime) - tseries(1) )/( ntime - 1 )
    • -tr=+/-3, the least-squares line from the time series is removed before time filtering.

    If -tr=-1, -2 or -3, the mean, drift or least-squares line are reintroduced post-filtering, respectively.

    For other values of the -tr= argument, nothing is done before or after filtering.

    If the -p= argument is present, the pre-filtering and post-filtering processing is applied to each time segment, separately.

    The -tr= argument must be an integer and the default value for the -tr= argument is 0.

  11. The -win= argument controls the form of the window, which will be convolved with the ideal filter response. By default, a Hamming window is used (e.g., -win=0.54).

    Set -win=0.5 for using a Hanning window or -win=1. for a rectangular window (e.g., the “ideal” filter).

    The -win= argument is a real number greater or equal to O.5 and less or equal to 1..

  12. The -mi=missing_value argument specifies the missing value indicator for the output variable in the output_netcdf_file.

    If the -mi= argument is not specified, the missing_value is set to 1.e+20.

  13. The -double argument specifies that the results are stored as double-precision floating point numbers in the output NetCDF file.

    By default, the results are stored as single-precision floating point numbers in the output NetCDF file.

  14. The -hdf5 argument is allowed only if the NCSTAT software has been compiled with the _USE_NETCDF4 CPP macro (e.g., -D_USE_NETCDF4) and linked to the NetCDF 4 library or higher.

    If this argument is specified, the output_netcdf_file will be a NetCDF-4/HDF5 format file instead of a NetCDF classic format file. However, this argument is recognized in the procedure only if the NCSTAT software has been built with the _USE_NETCDF4 CPP macro.

  15. It the time series has a seasonal or diurnal cycle, use comp_stl_1d to remove the pure harmonic components from the time series before filtering.

  16. It is assumed that the data has no missing values.

  17. Duplicate parameters are allowed, but this is always the last occurrence of a parameter which will be used for the computations. Moreover, the number of specified parameters must not be greater than the total number of allowed parameters.

  18. For more details on windowed filtering, see

Outputs

comp_fftfilter_1d creates an output NetCDF file that contains the filtered time series estimated from the time series associated with the input NetCDF variable. The output NetCDF dataset contains the following NetCDF variable (in the description below, ntime is the selected number of time observations) :

  1. netcdf_variable_filt(ntime) : the filtered time series for the time series associated with the input NetCDF variable.

Examples

  1. For windowed filtering a real monthly time series between 18 and 30 months (e.g., biennial time scale) from a NetCDF variable called sst extracted from the file sst.monthly.nino34.nc, which includes a monthly time series, and store the results in the NetCDF file qbo_sst_nino34.nc, use the following command :

    $ comp_fftfilter_1d  \
  -f=sst.monthly.nino34.nc \
  -v=sst \
  -pl=18 \
  -ph=30 \
  -o=qbo_sst_nino34.nc
Flag Counter