THD+N measurement with AES17-2015 notch filter

[RJA4000]

Hello

On the newest releases, REW added a "Use AES17-2015 standard notch" checkbox.

This was done, once again, to try and mimic Audio Precision results as close as possible
(In a similar attempt, they added Dolph-Chebyshev 200 windowing. You did as weel: Thanks for that !)

This hardly makes a difference in normal meausrements.
But when using a hardware notch filter to retrieve the very last tenths of DB SINAD on very performant DAC, this has a big impact.

Is that something we could achieve with MI ?
Do I just need a FCF file for this ?
Or do you need to change THD+N is computed for that notch result specifically ?

[VirtinsTech]

Thanks for the information.

According to REW's manual, "...if the 'Use AES17-2015 standard notch' option is selected the fundamental power for THD and THD+N calculations will be the power within a one octave span around the fundamental frequency. If that option is not selected it will be the power in the main lobe of the fundamental. When the fundamental level approaches the noise floor using the standard notch will produce a much higher figure than the fundamental main lobe, in those cases it is better not to use the standard notch...".

It does not seem to be good to use this option for the following reasons:

1. Octave bands do not have a fixed bandwidth. They become wider as frequency increases. In contrast, the main lobes of the fundamental and harmonics do have a fixed bandwidth for a fixed FFT size (The bandwidth is reduced as the FFT size increases). Their widths do not change with frequency. In THD calculation, it is obviously "unfair" if only the fundamental power is calculated based on one octave span around it while the harmonic powers are still calculated based on the main lobe bandwidths. On the other hand, it is also not correct if the harmonic powers are calculated based on one octave span around them because the octave span will be too wide at high frequencies (see table below).

1-1_OctaveBands.png (27.88 KiB) Viewed 30746 times

2. It is quite understandable to use this option in THD+N calculation to mimic the hardware method, however, in THD+N vs Frequency measurement, the fundamental frequency will be stepped from 20Hz to 20kHz. Imaging how wide one octave span will be at 20kHz. Even in THD+N @1kHz measurement, one octave span would be from 707Hz to 1414Hz, which is very wide compared with the main lobe method. The noise power inside that range will all be treated falsely as the power of the fundamental.

Having said the above, in Multi-Instrument, it is possible to calculate THD+N with AES17-2015 notch filter like REW using "Energy in user defined frequency bands" function as follows:

1. Define two frequency bands, one for the total energy, the other for the fundamental energy, e.g. 20Hz~20kHz, 707Hz~1414Hz

1.png (25.34 KiB) Viewed 30746 times

2. Define a UDDP using the following formula to calculate THD+N:
10*log10(pow(10,[fBand1RMS_A(EU)]/10)-pow(10,[fBand2RMS_A(EU)]/10))/pow(10,[fBand1RMS_A(EU)]/10)
That is: 10*log10([energy in band1] - [energy in band2])/[energy in band1])

2.png (16 KiB) Viewed 30746 times

The following picture shows an example of THD+N measurement. The THD+N measured by the AES17-2015 notch filter method is -82.75 dB while -82.58 dB is measured by the normal method. The former is lower because the bandwidth of the notch filter at the fundamental is much wider and the resulting fundamental power contains the contribution from noises in that octave band.

3.png (125.39 KiB) Viewed 30746 times

[RJA4000]

Big thanks for your time answering this !

Actually, I was conducting some investigations myself and also found a way to achieve the same:
Using "Energy in Band", I was able to compute D+N Level and "SINAD with AES-17 Notch)
(both on the bottom line)

D+N = 10*LOG10((10^([fBand1RMS_A(EU)]/10)+10^([fBand3RMS_A(EU)]/10)))
and SINAD = -(10*LOG10((10^([fBand1RMS_A(EU)]/10)+10^([fBand3RMS_A(EU)]/10)))-[fBand4RMS_A(EU)])

2022-06-27 12_36_12-Greenshot.png (215.12 KiB) Viewed 30740 times

I then get the exact same value than REW.
Problem is that I then loose all other informations regarding Harmonci levels and so on...


This way of working is only important because while using a notch filter, followed by an amp and a compensation file to remove its effect, the "Skirt" is wider, and therefore we have more of the fundamental in the noise than just due to the window lobes.
But that's the only way to get this level of THD+N measurements...

That's a workaround, in my opinion, that was probably introduced by AES standard in 2015 to allow this (especially for Audio Presision hardware, which is using this kind of technique)
So having both options would be great.
Ideally, adding a few additional DDPs for this method ("SINAD AES-17", "SNR AES-17", or so)

Any idea how to solve it ?

[RJA4000]

By the way, I posted this

https://www.audiosciencereview.com/foru ... st-1230502

I hope I didn't write anything stupid...

[VirtinsTech]

This way of working is only important because while using a notch filter, followed by an amp and a compensation file to remove its effect, the "Skirt" is wider, and therefore we have more of the fundamental in the noise than just due to the window lobes.
But that's the only way to get this level of THD+N measurements...

If a proper window function is chosen, the loss of fundamental energy due to the limited calculation bandwidth (BTW, MI has its own way to calculate this bandwidth, somewhat similar to but not exactly as the main lobe method) can be ignored, as illustrated in the following example.

FundamentalBandwidth.png (72.74 KiB) Viewed 30719 times

In the above example, the total energy calculated from 0Hz~24kHz is 707.10678679663386 mV (RMS), the fundamental energy f1RMS_A calculated by MI is 707.10678679663374 mV (RMS) which is the same as the energy calculated from 999Hz ~1001Hz. The leakage energy is thus only 0.0000130271 mV (RMS). If the leakage energy is treated as noise and distortion, then the THD+N would be -155dB (from 0~24kHz). The THD+N calculated in the above screenshot is -149.86 dB (from 20Hz~20kHz). This example shows that the leakage of the fundamental by the normal THD+N calculation method can hardly affect any actual THD+N measurement (which is usually much higher than -150dB). On the other hand, with the narrow fundamental energy calculation bandwidth (<2Hz here) compared with the AES17-2015 notch filter (707Hz~1414Hz), much more noise energy is separated from the fundamental energy, resulting in improved measurement accuracy.

[VirtinsTech]

By the way, I posted this...

Thanks for the posting. BTW, the fundamental energy calculation bandwidth increases as FFT size decreases. In case the fundamental skirt is wide due to any reasons, reduce the FFT size.

[RJA4000]

I'm still struggling with this one.

As an example, I compare the results for the same DAC with 2 different ADCs, and compared results from my "home made " SINAD to the "Official" SINAD

2022-07-18 12_52_44-AES-17 Notch debug.png (109.18 KiB) Viewed 30299 times

(X axis are Vrms - sorry for that, it was just a "quick and dirty" plot)

There is some coherent results:
When DAC and ADC are in the same machine (the RME), there is no difference of result between both methods, which is to be expected.
When the ADC is separate, we see a small gap... until it suddenly explodes at between -3.0dBFS and -2.5dBFS...

So what's going on at -2.5dBFS ?

2022-07-18 17_45_07-MI Notch -2.5dBFS.png (158.31 KiB) Viewed 30299 times

I compared with REW (AES-17 notch)

2022-07-18 17_51_27-Cosmos ADC PCM32_384 1 on ASIO4ALL v2 at 48 kHz, 32.768-point Spectrum, Dolph-Ch.png (189.42 KiB) Viewed 30299 times

The main difference is on the noise.

You see my bands here:
;A-Frequency Band (Hz), A-RMS (dBFS)
706.055Hz ~ 1410.64Hz, -2.61886
1984.86Hz ~ 2006.84Hz, -145.616
2982.42Hz ~ 3004.39Hz, -129.26
3979.98Hz ~ 4001.95Hz, -137.651
4977.54Hz ~ 4999.51Hz, -139.163
5975.1Hz ~ 5997.07Hz, -148.723
6972.66Hz ~ 6994.63Hz, -146.379
7970.21Hz ~ 7992.19Hz, -150.739
8967.77Hz ~ 8989.75Hz, -144.307
9965.33Hz ~ 9987.3Hz, -149.503
10962.9Hz ~ 10984.9Hz, -147.226
11960.4Hz ~ 11982.4Hz, -151.459
12958Hz ~ 12980Hz, -150.691
13955.6Hz ~ 13977.5Hz, -148.718
14953.1Hz ~ 14975.1Hz, -144.512
15950.7Hz ~ 15972.7Hz, -148.078
16948.2Hz ~ 16970.2Hz, -151.32
17945.8Hz ~ 17967.8Hz, -151.996
18943.4Hz ~ 18965.3Hz, -148.199
19940.9Hz ~ 19962.9Hz, -150.035
20.5078Hz ~ 19999.5Hz, -2.61886

The last one being full bandwidth.

(By the way: is the upper frequency bin included in the result for the band or not ? It seems it's not.
As an example 11960.4Hz ~ 11960.4Hz gives me -[infinity] result.)

I compute noise level as follows:
10*LOG10(
10^([fnRMS_A(EU)_Array[20]]/10)
-10^([fnRMS_A(EU)_Array[0]]/10)
-10^([fnRMS_A(EU)_Array[1]]/10)
-10^([fnRMS_A(EU)_Array[2]]/10)
-10^([fnRMS_A(EU)_Array[3]]/10)
-10^([fnRMS_A(EU)_Array[4]]/10)
-10^([fnRMS_A(EU)_Array[5]]/10)
-10^([fnRMS_A(EU)_Array[6]]/10)
-10^([fnRMS_A(EU)_Array[7]]/10)
-10^([fnRMS_A(EU)_Array[8]]/10)
-10^([fnRMS_A(EU)_Array[9]]/10)
-10^([fnRMS_A(EU)_Array[10]]/10)
-10^([fnRMS_A(EU)_Array[11]]/10)
-10^([fnRMS_A(EU)_Array[12]]/10)
-10^([fnRMS_A(EU)_Array[13]]/10)
-10^([fnRMS_A(EU)_Array[14]]/10)
-10^([fnRMS_A(EU)_Array[15]]/10)
-10^([fnRMS_A(EU)_Array[16]]/10)
-10^([fnRMS_A(EU)_Array[17]]/10)
-10^([fnRMS_A(EU)_Array[18]]/10)
-10^([fnRMS_A(EU)_Array[19]]/10)
)

And SINAD as
-10*LOG10(
10^([fnRMS_A(EU)_Array[20]]/10)
-10^([fnRMS_A(EU)_Array[0]]/10)
)
+[fnRMS_A(EU)_Array[20]]
(Could do with [fnRMS_A(EU)_Array[0]], but result is the same)

Initially, I was using the "Official" THD value to compute the sum of distortions level, but a you see that makes little difference (0.1dB distortion difference)
EDIT: Actually, it does !
Writing the above I suddenly had a doubt and wanted to check.
At those very low levels, a tiny difference in distortion may have a big impact on the result.
That doesn't explain it all, for sure. But still, it makes my quest even more difficult, I'm afraid.

Any idea what I do wrong here ?

[RJA4000]

Here-attached are more details

[VirtinsTech]

There is some coherent results:
When DAC and ADC are in the same machine (the RME), there is no difference of result between both methods, which is to be expected.
When the ADC is separate, we see a small gap... until it suddenly explodes at between -3.0dBFS and -2.5dBFS...
So what's going on at -2.5dBFS ?

There are two methods to calculate Noise + Distortion, one is to calculate it directly, the other is to do it indirectly via Noise +Distortion = Total - Fundamental. These two methods are theoretically the same but when it comes to numerical computation, they can be different. One example was actually found by you back in 2019 at viewtopic.php?f=7&t=1382&start=10. When the Noise + Distortion is extremely small and both the Total and Fundamental are very big, the indirect method may not be able to maintain sufficient precision due to the truncation during numerical computation. This might be the reason here when the signal level goes very high with "home made" notch method (i.e. the red one).

When the DAC and ADC do not share the same sampling clock, the unsynchronized clock jittering might make the skirt of the fundamental a bit wider. MI should still be able to handle this situation.

You see my bands here:
;A-Frequency Band (Hz), A-RMS (dBFS)
706.055Hz ~ 1410.64Hz, -2.61886
1984.86Hz ~ 2006.84Hz, -145.616

The bandwidth you used for harmonics is about 22Hz. MI internally uses 28Hz here, but this makes little difference in harmonic level calculation.

(By the way: is the upper frequency bin included in the result for the band or not ? It seems it's not.
As an example 11960.4Hz ~ 11960.4Hz gives me -[infinity] result.)

If the start frequency and end frequency are the same, then the bandwidth is zero, thus zero energy can be contained inside.

I compute noise level as follows:
10*LOG10(
10^([fnRMS_A(EU)_Array[20]]/10)
-10^([fnRMS_A(EU)_Array[0]]/10)
-10^([fnRMS_A(EU)_Array[1]]/10)
-10^([fnRMS_A(EU)_Array[2]]/10)
-10^([fnRMS_A(EU)_Array[3]]/10)
-10^([fnRMS_A(EU)_Array[4]]/10)
-10^([fnRMS_A(EU)_Array[5]]/10)
-10^([fnRMS_A(EU)_Array[6]]/10)
-10^([fnRMS_A(EU)_Array[7]]/10)
-10^([fnRMS_A(EU)_Array[8]]/10)
-10^([fnRMS_A(EU)_Array[9]]/10)
-10^([fnRMS_A(EU)_Array[10]]/10)
-10^([fnRMS_A(EU)_Array[11]]/10)
-10^([fnRMS_A(EU)_Array[12]]/10)
-10^([fnRMS_A(EU)_Array[13]]/10)
-10^([fnRMS_A(EU)_Array[14]]/10)
-10^([fnRMS_A(EU)_Array[15]]/10)
-10^([fnRMS_A(EU)_Array[16]]/10)
-10^([fnRMS_A(EU)_Array[17]]/10)
-10^([fnRMS_A(EU)_Array[18]]/10)
-10^([fnRMS_A(EU)_Array[19]]/10)
)

And SINAD as
-10*LOG10(
10^([fnRMS_A(EU)_Array[20]]/10)
-10^([fnRMS_A(EU)_Array[0]]/10)
)
+[fnRMS_A(EU)_Array[20]]
(Could do with [fnRMS_A(EU)_Array[0]], but result is the same)

When the signal level is very high and noise and distortion level is very low, direct calculation method will be more accurate.

[VirtinsTech]

Comparison between MI’s official results and REW (AES-17 notch) results in your two screenshots

1. THD: -124.9 dB (MI) vs -125.0 dB (REW with AES-17 notch)

2. THD+N: -118.4 dB (MI) vs -118.8 dB (REW with AES-17 notch)

3. Noise Level: -121.4 dBFS (MI) vs -122.6 dBFS (REW with AES-17 notch)

The small differences in the above parameters are attributed to the fact that the fundamental calculation bandwidth is much wider in AES-17 Notch (707Hz wide) than in MI (28Hz wide here), resulting in increased fundamental level and reduced noise level, which may not be correct in most of the cases.

The THD value and the fundamental level are almost the same in the two cases. However, it is interesting to note that REW is about 3~4 dB higher in almost all displayed harmonics except the 8th one.