Any kind of sound processing (e.g. finding similar songs by content) requires the uncompressed data, e.g. for Fourier transforms (FFT), MFCC extraction etc. There are tons of different audio compression algorithms and file formats (MP3, WMA, MPA, CDA, AIF, OGG, AC3, M4A, FLAC, RMI, ...), each based on a different codec to decode the sound data. Finding a library to decode all these formats seems like an impossible task. There are cross-platform solutions like FFMPEG, OPENCV(?) and partial solutions like MPG123 (which only decodes MP3).
If you develop on Windows on the other hand, and don't care about cross platform, and target windows vista and later, there's Microsoft Media Foundation (MMF), which is the evolution of DirectShow of yore. Media Foundation is a combination of C++ API and several COM objects, that deal with reading, decoding and playing media files (audio and video). The full MMF is quite complex, dealing with topologies, sessions and transformations. Luckily for us, MMF's source reader objects were designed just for the task at hand, loading any media file and mapping the correct installed MFT codec to extract pure audio (it also decodes video frames). The procedure is quite easy:
Minimum requirements: windows Vista or later