FFT analysis: SNR and Noise level vary with FFT Window size

[RJA4000]

Hello

I noted a strange behaviour:
When I keep everything else the same, the SNR and NL are varying with the FFT Window size.

That doesn't look right to me.

I posted the analysis here:
https://www.audiosciencereview.com/forum/index.php?threads/fft-size-impact-on-noise-level-in-virtins-multi-instrument-3-8.9194/#post-236827,
since it was following a discussion we had on that forum.

Could you please have a look and tell me what's wrong ?
Can perfectly be me missing something

[VirtinsTech]

Thank you for posting MI results on other forums.

FFT size affects the measurement accuracy of THD, THD+N, SNR, SINAD and Noise Level in a few ways. Every aspect can be a long story

1. DC residual artefact

This is not the most important effect. It is placed as #1 here to address the issue in your first screenshot in the above link.

Assuming a sinewave signal contains no DC offset, its long-time mean value will approach zero. If a FFT segment contains a non-integer number of cycles of the signal, the mean value within that FFT segment will not be zero and an artificial DC residual is thus created. This DC residual artefact will die out gradually as FFT size increases. On the other hand, if a FFT segment contains an integer number of cycles of the signal, the mean value within that FFT segment will always be zero. For example,

Example 1:
If [FFT Size]=[Record Length]=2048, [Sampling Rate] =48000Hz, [Signal Frequency]=1000Hz, then a FFT segment contains 2048/48000*1000 = 42.6667 (non-integer) signal cycles. It can be seen from the following picture that the DC residual is measured 5.79873 mV in the oscilloscope window, and there is a strange “hump” covering the low frequency region including DC in the spectrum analyzer window. This hump is a result of the convolution between the artificial DC residual and the window function used in the spectrum analyzer (Note: multiplication in time domain is equivalent to convolution in frequency domain). It resembles the main lobe of Kaiser 6 window.

DCresidual1.png (103.88 KiB) Viewed 68299 times

Example 2:
If [FFT Size]=[Record Length]=2048, [Sampling Rate] =48000Hz, [Signal Frequency]=1007.8125Hz, then a FFT segment contains 2048/48000*1007.8125 = 43 (integer) signal cycles. It can be seen from the following picture that the DC residual is measured 0.00 uV in the oscilloscope window, i.e. virtually no DC residual artefact.

DCresidual3.png (117.39 KiB) Viewed 68291 times

The above two examples use simulated signals generated directly from the signal generator of MI without going through any hardware. The measured signal is thus “ideal”. Using simulated signals help to validate various software settings and learn the highest measurement accuracy achievable using these settings.

In real life, a signal can have a substantial DC offset (unlike the above DC residual artefact). This DC offset is usually not of interest in spectrum analysis. In the spectrum analyzer of MI, a ”Remove DC” option is provided via [Spectrum Analyser Processing]>”Intra-Frame Processing” to remove the DC component (i.e. the mean value measured in the oscilloscope) before FFT. The above two pictures were captured with this option ticked. The following picture was captured with this option unticked for Example 1. It can be seen that in the spectrum analyzer window, the “hump” covering the low frequency region including DC disappears. This may look sarcastic as “Remove DC” option actually adds more DC in Example 1. The reason is that the mean value in the oscilloscope is calculated without applying any window function. When a window function (other than Rectangle window) is applied, the fractional cycle at the end of the waveform that causes an artificial DC residual is already greatly suppressed, ticking “Remove DC” will actually add more artificial DC residual when a non-rectangle window is applied. In this particular case, unticking "Remove DC" works better. However, if the test signal contains a substantial DC offset, then “Remove DC” is still recommended, a larger FFT size or no-spectral leakage test frequency can be used to suppress or remove the DC residual artefact.

DCresidual2.png (99.32 KiB) Viewed 68299 times

Unticking “Remove DC” will cause little difference for Example 2.

[RJA4000]

Thanks
That's detailed explanation !

I see 1.
So I guess there will be a follow-up ?

[VirtinsTech]

2. Spectral Leakage

Spectral leakage occurs when a FFT segment contains a non-integer number of signal cycles. It can be suppressed by applying a window function and using a larger FFT Size (and Record Length).

A window function helps to converge most of the power of a sinusoidal signal into a few FFT bins around its peak, leaving only a tiny amount of power leaking to FFT bins farther away. The following screenshot and its closeup show the spectra of a 1 kHz sinewave with different window functions applied: Hann, Blackman Harris, Kaiser 6 and Rectangle (i.e. no window function).

THD_Window.png (103.95 KiB) Viewed 68197 times

THD_Window_ZoomIn.png (75.94 KiB) Viewed 68197 times

It can be seen that among them, Kaiser 6 is the best window function that has the least spectral leakage while Rectangle is the worst one with which all harmonics are submerged under the spectral leakage. Although it is possible to calculate audio parameters such as THD using Hann or Blackman Harris (for example, one can use a much wider bandwidth to compute the power of the fundamental when Hann window is used. But that also means that the noise power that falls into those FFT bins will be falsely taken into the fundamental’s account), the results will be compromised compared with those obtained using Kaiser 6. It is reasonable to expect that there will be some differences among the results using different window functions regardless what software calculation method is used after windowing. Only Kaiser 6 ~ Kaiser 10 and the like is recommended for THD measurement in MI.

Similarly, the larger the FFT size, the higher the frequency resolution (Note: [FFT frequency resolution]= [Sampling Rate]/[FFT size]), the more concentrated the sinusoidal power. The following screenshots shows the spectra of a 1kHz sinewave with different FFT sizes: 2048, 32768, 524288. The frequency resolutions are thus: 23.43Hz, 1.4648Hz, 0.091553Hz. The FFT bin bandwidth ratio is: 256: 16: 1. Assuming the noises are distributed uniformly along the frequency axis, the ratio of the noise power contained in each FFT bin is also: 256: 16: 1. In other words, there is a 12dB difference in the apparent noise floors in between among the three cases. This can be seen from the screenshot and its closeup below.

THD_Resolution.png (108.52 KiB) Viewed 68197 times

THD_Resolution_ZoomIn.png (78.81 KiB) Viewed 68197 times

The apparent noise floor changes with FFT size. This is why MI shows the total noise level which does not change with FFT size in the spectra of THD measurements. In contrast to the noises, the heights of the peaks of sinusoidal components remain almost unchanged with FFT size and thus become visible or more prominent above the apparent noise floor as FFT size increases. On the other hand, the widths of the peaks of sinusoidal components become narrower (thus sinusoidal power is more concentrated) as FFT size increases, separating the noise and signal more clearly. It is thus reasonable to expect that there will be some differences among the results using different FFT size regardless what software calculation method is used, and generally, the measurement accuracy will improve as FFT size increases. Otherwise there would be no point to increase FFT size. The differences caused by different FFT sizes (in a certain range, of course) should be small, though.


3. Clock Jitter
Clock Jitter appears as sidebands around the peaks of sinusoidal components. When FFT size is small, the shape of the fundamental peak is usually dominated by the window function used and the jitter sidebands are confined within the width of the fundamental peak determined by the window function. As FFT size increases, sampling duration increases, it is more likely to capture the slow clock drift in the spectrum. Meanwhile, as the width of the fundamental peak becomes narrower when FFT size increases, it is more likely for the jitter sidebands move out from the width of the fundamental peak determined by the window function. Some special arrangement has been implemented to cope with this situation in MI.

Having said the above, your measurement with FFT size = 1048576 on https://www.audiosciencereview.com/foru ... ost-236827 seems to have some issues. The spectrum pattern is a little bit different from all your other measurements (look at the skirt of the fundamental). Not sure whether there are some dropouts in the captured waveform or not, as this may be a resource-consuming and stressful test for your computer.

[RJA4000]

Hi

Thanks again for your detailed explanation.

So I have the answer to the first case:

Spectral leakage

FFT Size MI 2k Leak.jpg (245.65 KiB) Viewed 68097 times

It's pretty clear there is a DC offset due to spectral leakage.

Which we see fixed here:

FFT Size MI 2k No Leak.jpg (255.78 KiB) Viewed 68097 times

That's covered by your explanation #1

Second, it's also clear that the last case, with FFT size of 1024k is related to a PC performance issue:
As the processing takes longer, my PC is generating non-stable signal, and therefore we have lower performance.
That may appear also when I change window or do something else on the PC while the measurement is on-going.
That's not specific to Multi Instrument.
(What is specific though is that it seems to take longer for MI to process such a huge FFT window)
Obviously, I have to take a look at the PC and optimize the drivers and other background jobs.

That's what you highlighted at the end of your post, and I agree.

your measurement with FFT size = 1048576 on https://www.audiosciencereview.com/foru ... ost-236827 seems to have some issues. The spectrum pattern is a little bit different from all your other measurements (look at the skirt of the fundamental). Not sure whether there are some dropouts in the captured waveform or not, as this may be a resource-consuming and stressful test for your computer.

But what I still don't understand is the big different in Noise Level figure between, say, a FFT of 32768 and 131072.
We jump from -119dB to -123dB (unweighted).

MI Table.PNG (35.42 KiB) Viewed 68097 times

That doesn't look like "small differences" as you were describing here:

The apparent noise floor changes with FFT size. This is why MI shows the total noise level which does not change with FFT size in the spectra of THD measurements. In contrast to the noises, the heights of the peaks of sinusoidal components remain almost unchanged with FFT size and thus become visible or more prominent above the apparent noise floor as FFT size increases. On the other hand, the widths of the peaks of sinusoidal components become narrower (thus sinusoidal power is more concentrated) as FFT size increases, separating the noise and signal more clearly. It is thus reasonable to expect that there will be some differences among the results using different FFT size regardless what software calculation method is used, and generally, the measurement accuracy will improve as FFT size increases. Otherwise there would be no point to increase FFT size. The differences caused by different FFT sizes (in a certain range, of course) should be small, though.

The apparent noise floor is changing, for sure, but the SNR or Noise Level should not, or by a very limited amount.
Which is what you confirm here :

The apparent noise floor changes with FFT size. This is why MI shows the total noise level which does not change with FFT size in the spectra of THD measurements.

4dB or even 5dB difference doesn't look small.
(By the way, the Noise Level bar in the Amplitude Spectrum graph is really a cool feature)

I understand computing the noise is not easy, since you have this Window size effect on apparent noise.
Do you have any idea to explain that difference ?


(Thanks for your patience)

[VirtinsTech]

The total power (T) of the signal consists of three parts: fundamental of the signal (S), harmonic distortion (D=H2+H3+H4+...), noises (N), i.e. T = S + D + N. THD, THD+N, SNR, SINAD, NL are derived based on the measurement of these three parts. T can be readily calculated by summing up the power in all FFT bins in the specified frequency range of interest (e.g. 20Hz~20kHz). The power of a sinusoidal component (e.g. S, H2, H3, H4…) can be calculated from a few FFT bins around its peak (See figure below).

THDcalculation.png (43.52 KiB) Viewed 68021 times

The bandwidth (BW) that is used to calculate the power of a sinusoidal component reduces as FFT frequency resolution increases. As a result, the measurement accuracy increases. The BW is usually software algorithm dependent. The following two examples illustrate how BW can be selected and its impact on S,D and N calculation.

1. [FFT Size] = 2048, [Frequency Resolution] = 23Hz
THD = -115.02dB, THD+N = -111.28dB, SINAD = 111.28 dB, SNR =113.67dB, NL = -116.68 dBFS

FFT=2048-BW.png (84.78 KiB) Viewed 68021 times

The BW in this case is 1172Hz-820Hz=352 Hz which can be clearly seen from the width of the fundamental peak. This is pretty coarse as the noise that falls into this BW will be falsely taken as a part of the fundamental or harmonic power.


2. [FFT Size] = 2097152, [Frequency Resolution] = 0.023Hz
THD = -118.09dB, THD+N = -116.12dB, SINAD = 116.12 dB, SNR =120.50dB, NL = -123.52 dBFS

FFT=2097152-BW.png (85.71 KiB) Viewed 68021 times

The BW in this case is 1002.52Hz – 997.483Hz = 5Hz. The corresponding S=-3.02174408385 dBFS, as shown in the close-up of the fundamental peak.

FFT=2097152-BW-CloseUp.png (95.97 KiB) Viewed 68021 times

Now, if we use Kaiser 6’s main lobe width BW =1000.18Hz - 999.841Hz=0.339 Hz for the fundamental power calculation instead, then S=-3.02174409001 dBFS. The difference between the two methods is negligibly small (only 0.00000000616 dBFS) for the fundamental power calculation. However, this will cause big difference in noise level calculation, because the power levels in the frequency bands 997.483Hz ~ 999.841 Hz and 1000.18Hz ~ 1002.52Hz are -94.61dBFS and -94.42 dBFS respectively, much higher than the measured noise level -123.52 dBFS. In other words, a slightly-narrower-than-proper BW can falsely increase the noise level significantly. On the other hand, due to the nearly uniform distribution of noises, a slightly-wider-than-proper BW will be on the safer side and has minimum impact on the results. The picture also shows the power level in the frequency bands 20 Hz ~ 997.483 Hz and 1002.5Hz~20000Hz are -124.52 dBFS and -114.38 dBFS (note: this one contains both noise and harmonic distortion in this frequency band) respectively. The above power levels are calculated using [Spectrum Analyzer Processing]>"Energy in User Defined Frequency Bands" in MI.

The noise level difference in the above two examples is -116.68 dBFS - 123.52 dBFS = 6.84 dBFS. if you want Example 2 to measure the same noise level as Example 1, then the BW have to be reduced, which does not look correct from the spectrum, especially when a much wider BW is actually used in Example 1.

The skirts of the harmonics are usually buried under the noise floor and thus do not account for the above noise level difference.

In summary, the measurement accuracy will be improved with a larger FFT size, because the larger the FFT size, the higher the frequency resolution, the narrower the BW required, the better segregation of S, D and N. Whether the improved measurement accuracy will lead to better or worse audio parameters depends on the spectrum details, in particular the skirt of the fundamental.

[VirtinsTech]

The above example with FFT size = 2097152, SR=48kHz, sampling duration =2097152/48000=43.69 s, the display takes about 48 s to refresh, therefore the computing time is about 48-43.69 = 4.3 s. The computer used is Dell XPS 8900, Processor Intel Core i7-6700 CPU @ 3.40GHz, 8GB RAM, Windows 10 Pro, version 1809.

[RJA4000]

Hello

I've performed some more tests.

First, I've redone same tests.
Then I've checked with varied level.

Here is the variation of Noise Level:

FFT Size and Level_.png (76.08 KiB) Viewed 68005 times

(Note that -2dBFS 1024k size measurement was not stable and should be removed)

As one can see, the Noise level varies quite a lot with the FFT Window size when level is at -2.4dBFS (pretty similar to my previous measurements)
But it's stable when you lower the level.

Does that mean that the distortion components (which are higher with higher level) are taken into account differently as the FFT window size changes ?
Or is it the fundamental removal which impacts the NL differently ?

[RJA4000]

Another interesting thing is the variation with the number of averaged frames:

FFT Avg_.png (18.14 KiB) Viewed 68005 times

I guess this is normal.
Note that all other measurements were done with 4 Averages values

[RJA4000]

And, last but not least, I had MI crashing systematically in following conditions:

High FFT Window Size (2M in my case), when I start the generator then the ADC, MI crashes just after displaying a level in the top-right bargraph.
That is, when it starts computing results, I guess.

When I have the generator running and the ADC running too, if I change the window size, then it works without issue.
Annoying for automation...

Here is the Eventviewer message:

Log Name: Application
Source: Application Error
Date: 12/10/2019 12:49:14
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: XYZXYZXYZ
Description:
Faulting application name: MIs.exe, version: 3.8.5.2, time stamp: 0x5c2ec2f6
Faulting module name: MIs.exe, version: 3.8.5.2, time stamp: 0x5c2ec2f6
Exception code: 0xc0000005
Fault offset: 0x00151c76
Faulting process id: 0x41c8
Faulting application start time: 0x01d580ea7c482ac3
Faulting application path: C:\VIRTINS Multi-Instrument 3.8\MIs.exe
Faulting module path: C:\VIRTINS Multi-Instrument 3.8\MIs.exe
Report Id: d5b4b931-5227-4052-b649-8824ecc9a0d3
Faulting package full name:
Faulting package-relative application ID:

And here is the PC config:

PC Infos.PNG (97.83 KiB) Viewed 68005 times