A Perceptual Evaluation of Music Real-Time Communication Applications

Abstract

Music Real-time Communication applications (M-RTC) enable music making (musiking) for musicians simultaneously across geographic distance. When used for musiking, M-RTC such as Zoom and JackTrip, require satisfactorily received acoustical perception of the transmitted music to the end user; however, degradation of audio can be a deterrent to using M-RTC for the musician. Specific to the audio quality of M-RTC, we evaluate the quality of the audio, or the Quality of Experience (QoE), of five network music conferencing applications through quantitative perceptual analysis to determine if the results are commensurate with data analysis. The ITU-R BS.1534-3 MUlti Stimulus test with Hidden Reference and Anchor (MUSHRA) analysis is used to evaluate the perceived audio quality of the transmitted audio files in our study and to detect differences between the transmitted audio files and the hidden reference file. A comparison of the signal-to-noise ratio (SNR) and total harmonic distortion (THD) analysis to the MUSHRA analysis shows that the objective metrics may indicate that SNR and THD are factors in perceptual evaluation and may play a role in perceived audio quality; however, the SNR and THD scores do not directly correspond to the MUSHRA analysis and do not adequately represent the preferences of the individual listener. Since the benefits of improved M-RTC continue to be face-to-face communication, face-to-face musiking, reduction in travel costs, and depletion of travel time, further testing with statistical analysis of a larger sample size can provide the additional statistical power necessary to make conclusions to that end.