Sound recognition is one of the main application areas of artificial intelligence. Various sound recognition techniques exist and can be used for recognizing speeches, music, environmental sounds, animal sounds, air-borne or structure-borne machine sounds, etc. These techniques contain in common signal pre-processing, feature extraction and feature matching / classification algorithms from a general perspective. However, they differ in implementation in different applications. For example, there are a few good automatic speech recognition software products and APIs, however, they just cannot be used directly in animal sound classifications.
Specific sound recognition applications usually require custom programming. Virtins Technology has been actively involved in this line of business for years. Some projects that we have completed are:
Through years of researches and driven by actual projects, we have developed a suite of sound recognition API as well as VC++ and VC# sample programs. The API is called Sound Monitoring & Matching API. Its features include:
The freeware Sound-Similar (Free Version) below is developed using this API. Contact us for details if interested.
Last but not the least, some applications may look as if they require advanced sound recognition techniques, but their tasks could actually be performed successfully through proper configuration of Multi-Instrument. Example Video: Identification of Lighter Sounds in a Controlled Environment.
Sound-Similar (Free Version) is a lightweight software application that measures the perceptual similarity between two WAV files in Linear PCM format, the most commonly used format in WAV files. The degree of similarity is represented by a similarity score in percentage, from 0% to 100%. The software does not compare the two digital files bit by bit. Nor does it measure the resemblance of the two waveforms. The similarity is rated based on human auditory perception through advanced time, frequency, and time-frequency domain analyses. The resultant similarity score can be used for sound classification as well as perception-based sound quality check.
The two WAV files to be compared can have different sampling rates, bit depths, and one or two channels of data. If the file is stereo, then average values of the two channels will be used for comparison. Sound volume difference does not affect the similarity measurement unless it is too low to maintain the perceptible sound quality.
The frequency ranges of different classes of sounds, such as human speeches, music and environmental sounds, may be different. Sound-Similar allows the user to specify the comparison frequency range in order to increase the scoring accuracy. It is possible to extend this range to infrasonic or ultrasonic region.
There are two comparison modes: (1) Full Length vs Full Length, and (2) Full Length of the Shorter vs Partial Length of the Longer. In both modes, the two WAV files can have different lengths in time. Mode 2 can be used to check whether the shorter is a part of the longer and, if YES, where the shorter is located in the longer. The minimum length of the sound should be at least 50 ms and greater than the reciprocal of the frequency lower limit. There is virtually no restriction on the upper limit of the sound length as long as the computer memory allows.
There are three similarity modes: (1) Disregard Transmission Channel Difference; (2) Penalize for Transmission Channel Difference; (3) Penalize for Transmission Channel Difference and Heavily for any Difference in Content Delivery Speed. Version 1.0 supports Similarity Mode 1 only. Similarity Modes 2 and 3 were added in Version 1.1.
The similarity scoring algorithm in the software has been optimized for general-purpose use. A similarity score below a few percent usually indicates that the two sounds are totally different, while a score from a few percent to 100% often shows that the two are alike with a varying degree of similarity. In other words, not only can Sound-Similar classify sounds but also perform quality check against standard samples.
Sound-Similar was programmed using Sound Monitoring and Matching (SMM) API developed by Virtins Technology. The underlying API is much more flexible in setting various comparison parameters and has more advanced options such as background noise removal, short-time noise and distortion detection and different scoring methods.
Both are 2-second DTMF tone "1" consisting of 697Hz and 1209Hz, sampled at 44.1kHz and 16 bits. The former was generated digitally by the signal generator of Multi-Instrument and thus is ideal without noises. The latter was recorded by a microphone from the same tone output by a speaker and thus contains substantial background noises. The volume difference is about 8 dB. The following picture shows the comparison of their waveforms, spectra and spectrograms. The measured similarity score is 99.5% thanks to the anti-noise setting inside Sound-Similar Free. Thus the answer is POSITIVE (i.e. not FALSE NEGATIVE).
The former is 2-second DTMF tone "2" consisting of 697Hz and 1336Hz, sampled at 44.1kHz and 16 bits. The latter is 2-second DTMF tone "1" consisting of 697Hz and 1209Hz with the same sampling parameters. The former was generated digitally by the signal generator of Multi-Instrument and thus is ideal without noises. The latter was recorded by a microphone from the sound output by a speaker and thus contains background noises. The volume difference is about 8 dB. The following picture shows the comparison of their waveforms, spectra and spectrograms. The measured similarity score is 12.1% and thus the answer is NEGATIVE (i.e. not FALSE POSITIVE). It should be noted that DTMF tones are quite similar to one another when perceived by humans. If two different DTMF tones have one frequency component in common, their similarity score can go up to a few tens of percent. However, if the two DTMF tones have two frequency components in common (i.e. same DTMF tones), then their similarity score will be very close to 100%.
This test is similar to the above Test 1(1) except that the volume of the first one is 60dB lower (i.e. 1/1000) than that of Test 1(1) or 52dB lower than the second one here. The similarity score is 96.6%, dropped only a little bit from 99.5% in Test 1(1), showing that the volume difference has little impact on the similarity score.
This test is similar to the above Test 1(1) except that the first one is 10-second long here, five times as long as the latter. The similarity score is 99.4%, almost unchanged compared with Test 1(1). It should be noted that the longer one here contains the same stationary signal across its full length.
The former is a 2-second sound from a high voltage transformer while the latter is a 2-second sound emitted by a partial discharge event in a substation. The former is quite stationary but the latter is not quite so. The measured similarity score is 0.6%. In other words, the partial discharge sound can be differentiated from the hum. It should be noted that the lower limit of the comparison frequency range was set to 30Hz to take into account the 50Hz or 60Hz hum in comparison.
Both are 2-second partial discharge sounds in a substation. The measured similarity score is 84.5%. In other words, the two sounds can be classified under the same tag. It should be noted that the lower limit of the comparison frequency range was set to 30Hz to take into account the 50Hz or 60Hz hum in comparison.
Both were recorded from the same music clip but with different speaker and microphone pairs and different background noises and echoes. The measured similarity score is 90.9%.
Both were recorded from the same music clip but with different speaker and microphone pairs and different background noises and echoes. The measured similarity score is 91.5%.
i. Piano-1-1-SR44100-16Bit-10s.wav vs Piano-2-1-SR44100-16Bit-10s.wav: similarity score 0.5%
ii. Piano-1-1-SR44100-16Bit-10s.wav vs Piano-2-2-SR44100-16Bit-10s.wav: similarity score 0.5%
iii. Piano-1-2-SR44100-16Bit-10s.wav vs Piano-2-1-SR44100-16Bit-10s.wav: similarity score 0.6%
iv. Piano-1-2-SR44100-16Bit-10s.wav vs Piano-2-2-SR44100-16Bit-10s.wav: similarity score 0.5%
Both were recorded from the same song clip but with different speaker and microphone pairs and different background noises and echoes. The measured similarity score is 71.9%.
Both were recorded from the same song clip but with different speaker and microphone pairs and different background noises and echoes. The measured similarity score is 83.5%.
i. Song-1-1-SR44100-16Bit-10s.wav vs Song-2-1-SR44100-16Bit-10s.wav: similarity score 0.3%
ii. Song-1-1-SR44100-16Bit-10s.wav vs Song-2-2-SR44100-16Bit-10s.wav: similarity score 0.2%
iii. Song-1-2-SR44100-16Bit-10s.wav vs Song-2-1-SR44100-16Bit-10s.wav: similarity score 0.2%
iv. Song-1-2-SR44100-16Bit-10s.wav vs Song-2-2-SR44100-16Bit-10s.wav: similarity score 0.3%
Both were recorded from the same speech clip ("Because these two really sounds similar!") but with different speaker and microphone pairs and different background noise levels and echoes. The measured similarity score is 80.5%.
The former is a speech clip "Because these two really sound similar!" while the latter is a speech clip "Because these two really sound different!". That is, only the last words are different. The measured similarity score is 60.5%.
The former was a speech clip "Because these two really sound similar!" while the latter was a speech clip "Because these two sound really similar!". That is, only the positions of two words are swapped, which is in a way equivalent to a two-word difference. The measured similarity score is 9.7%.
The former was a speech clip "Because these two really sound similar!" while the latter was a speech clip "Because these two sound similar really!". That is, only the positions of three words are changed, which is in a way equivalent to a three-word difference. The measured similarity score is 1.4%.
The two files were recorded with different speaker and microphone pair and different background noises and echoes. The former contains a 1-second excerpt of a song while the latter contains a 20-second excerpt of the same song. The comparison mode is set to "Full Length of the Shorter vs Partial Length of the Longer". The result shows that the sound from 14.477 s to 15.477 s in the second file is similar to the entire sound in the first file with a similarity score of 55.2%.
