Our free stuff

Software tests → MP3 vs Ogg Vorbis

Summary

• Abstract
• Objectives
• Method
• Results
• Conclusions
• Appendixes

Abstract

We compared the MP3 format to Ogg Vorbis. We were not able to draw precise conclusions. However, the Ogg q4 format seemed to provide the best compression (no audible difference compared to higher qualities and bitrates).

Objectives

We wanted to assess the improvements brought by Ogg compared to MP3, and to precise the differences between bitrates for both MP3 and Ogg.
We used a constant bitrate (CBR) for MP3, because it's mainly used this way. We used a variable bitrate (VBR) for Ogg for the same reason.

Method

We didn't want to compare encoding speeds, so we won't detail our machine and running processes.

The music : we used Jean Michel Jarre's Oxygene 4 (Images - The Best Of Jean Michel Jarre - Track #01) to create a wav file starting at second #20 and finishing at second #43.

The softwares :

• We used Audiograbber 1.83 to rip the track.
• We cut it with Windows' recorder to keep only the wav from 20 s to 43 s.
• We used LamedropXPd V.1.3.3 using LAME 3.96.1 (found on RareWares) to compress the file into MP3 96, 128, 160, 192, 224, 256 and 320 kbps
• We used oggdropXPd V.1.7.11 using libVorbis v1.1.0 (found on RareWares) to compress the file into Ogg q -1, 0, 1, 2, 3, 4, 5 and 6
• We used Audacity 1.2.3 to analyze the spectra of all audio files (original, MP3s and Oggs)
• We imported spectrum analysises into Excel (sorry we tried with OpenOffice but it was way to slow) in order to compare them.

Spectrum analysis parameters :

• FFT Size : 1024
• Smoothing window : Hanning

The original file was 4057988 Bytes (3.86 MiB).

Results

We did a VBR compression into MP3, and we noticed it was (around twice) slower than MP3 CBR. We did a CBR compression into Ogg, and we noticed it was (around twice) slower than Ogg VBR. But this wasn't the objective of our test and we didn't explore it more. We only mention it for informational purpose.

Filesizes :

• MP3 :
  • 96 kbps : 277106 Bytes
  • 128 kbps : 369474 Bytes
  • 160 kbps : 461844 Bytes
  • 192 kbps : 554212 Bytes
  • 224 kbps : 646582 Bytes
  • 256 kbps : 738950 Bytes
  • 320 kbps : 923688 Bytes
• Ogg :
  • q -1 : 144096 Bytes (nominal bitrate : 45000 - real : 50111)
  • q 0 : 205430 Bytes (nominal bitrate : 64000 - real : 71441)
  • q 1 : 247703 Bytes (nominal bitrate : 80000 - real : 86142)
  • q 2 : 298286 Bytes (nominal bitrate : 96000 - real : 103732)
  • q 3 : 346010 Bytes (nominal bitrate : 112000 - real : 120329)
  • q 4 : 383713 Bytes (nominal bitrate : 128000 - real : 133441)
  • q 5 : 491547 Bytes (nominal bitrate : 160000 - real : 170942)
  • q 6 : 616051 Bytes (nominal bitrate : 192000 - real : 214239)

Before going further, let me remind you that the audible frequencies range from (approximately) 20 Hz to 18-20 kHz.

We analyzed the way the original audio spectrum was respected by the different audio compressions.
First, we drew the audio spectra and determinated when (eg, the frequency at which) the compressed spectra differed from the original file with less than 0.5 decibel (dB) ("good match"), with more than 0.5 and less than 3 dB ("approx match"), and with more than 3 and less than 10 dB ("crazy match"). Beyond a 10 dB loss, we considered the sound as lost.

(Quick interpretation : the higher the limits, the more lossless the compression is.)

Then, we determinated when the compressed spectra lost more than 1 dB for the first time, and when it gained more than 1 dB for the first time. The variation over 1 dB was not considered if it didn't occur at least twice in a 500 Hz interval.


We noticed that Ogg tended to increase frequencies with more than 0.5 dB and even more than 1 dB under quality 4 (not included). We are not audio professionals, so we don't know if this is really a lot, but it made it difficult for us to draw conclusions on the bitrate equivalences between MP3 and Ogg.

Here are the gain/loss compared to the original spectrum of the MP3 128 and Ogg q4 files.


To sum up :

• MP3 96 respected the original file until a 15 kHz frequency, then it lost the sound
• MP3 128 respected the original file until a 15.6 kHz frequency, then a poor sound until 17 kHz
• MP3 160 respected the original file until a 16.2 kHz frequency, then an average sound until 17.5 kHz
• MP3 192 respected the original file until a 17.4 kHz frequency, then a poor sound until 19.2 kHz
• MP3 224 respected the original file until a 17,6 kHz frequency, then a poor sound until 19.8 kHz
• MP3 256 did approximately as MP3 224
• MP3 320 respected the original file until a 17,6 kHz frequency, then an average sound until 18.6 kHz, then a poor sound until 19.8 kHz
• Ogg q-1 respected the original file until a 3.8 kHz frequency, then an average sound until 13.7 kHz
• Ogg q0 respected the original file until a 6 kHz frequency, then an average (but better than q-1) sound until 15.1 kHz
• Ogg q1 respected the original file until a 7 kHz frequency, then an average (but better than q0) sound until 15.8 kHz, then a poor sound until 16.4 kHz
• Ogg q2 respected the original file until a 9.3 kHz frequency, then a rather good sound until 15.5 kHz
• Ogg q3 respected the original file until a 11.4 kHz frequency, then a good sound until 17.1 kHz, then a poor sound until 17.7 kHz
• Ogg q4 respected the original file until a 15.8 kHz frequency, then a very good sound until 19.1 kHz
• Ogg q5 respected the original file until a 20 kHz frequency, then an average sound until 20.5 kHz
• Ogg q6 respected the original file until a 20 kHz frequency, then an average (better than q5) sound until 21 kHz, and some poor sound until 21.5 kHz

Conclusions

Considering an audible range from 20 Hz to 18 kHz, there is probably no audible difference between MP3 224 kbps and MP3 with higher bitrates, and there is probably no audible difference between Ogg q4 and Ogg with a higher quality.
We are not able to propose MP3-Ogg equivalences, because at an equivalent bitrate, Ogg preserves a wider range of frequencies than MP3 but tends to modify the original sound (even if this doesn't seem audible to us) more than MP3. We didn't expect such a result so we haven't been able to analyze it properly and as deeply as it should be.

It's also possible that our +/- 0.5 dB criterion was a bad criterion.

It's even harder to draw conclusions, because everyone has their own perception of sounds. We think that Ogg q4 is almost perfect to store audio, but our main concern with this quality is "what about compressions and recompressions ?", since it tends not to keep the original file as well as mp3.

This test should be continued in order to compare more precisely Ogg's distorsion to MP3's.

Appendixes

All spectra : spectra compared to the original file, 0 means no difference. Difference in dB. On all these graphs, the X axis is the frequency in Hz, linear, from 40 Hz to 22 kHz.