The most popular software tools are tested and compared in the challenging task of finding duplicate songs.
◪ Jump to the results →
How do you discover near-duplicate music?
The technology behind duplicate file software is quite simple, almost trivial. You just examine all the files in your hard disk byte-for-byte, and discover all 100% identical files. This is fast and safe, but will not work very well for your music file collection. Audio is a wave that can be saved in many different formats, compressions and bitrates. WAV files are uncompressed, FLAC are compressed but lossless, MP3 files are smallest because they do lossy compression (with a small imperceptible loss of quality). So a song that you or me would instantly recognize, is hard to tell looking at the raw bytes stored in the file. Audio fingerprinting algorithms rely on heavy mathematics; they "listen" to the music and seek patterns. Such fingerprints can be used to compare songs, and see through differences in sampling frequencies, loudness and stereo/mono channels, to identify musically identical files — as opposed to binary-identical.
Take for instance the following 2 music files. It is the same song, but the file size of one is almost double (320kbps bitrate), the duration of one is 4 seconds longer (probably on account of silence at start) and the ID3 tags are all over the place — still they represent the same song!
Name Size Genre Length Bit rate Year 02-Siva.mp3 6.01MB Alternative 00:04:25 190kbps 1991 The Smashing Pumpkins - Siva.mp3 10.0MB Mp3 00:04:21 320kbps 2001
What we need therefore is software that can find musical duplicates, allowing for differences in amplitude and recording quality and file formats (MP3,WMA,AIF,OGG,FLAC,AAC,APE...). Media files need a computer program that can understand the music, not the raw bytes stored in them. The bytes may differ, even the waveform itself slightly, but musically-aware software will detect the similarity, almost as a human listener would.
Most of the aforementioned 102 duplicate cleaners are not "musically savvy". They either rely on naive byte-comparison, or at best use ID3 tags like song title, artist and composer for comparing music files — but these tags are notoriously unreliable for downloaded music. I have narrowed it down to just seven (7) computer programs (for Windows 7/10/11) that analyze the music for similarity and will find musical duplicates, which in alphabetical order are:
Audio comparer by Bolide software [ACO]
Audio dedupe by MindGems [ADE]
Duplicate cleaner Pro by DigitalVolcano [DCL]
Duplicate Media Finder [DMF]
Duplicate music fixer by Systweak [DFI]
i-DeClone by Zabkat [IDE]
Similarity by GAR Software [SIM]
I will also consider another popular program called Wise duplicate finder, just to show you how "naive" duplicate finders are not up to the task of discovering musically duplicate songs. All these programs are shareware with free trial, so you would expect a decent quality in departments such as:
- Handle many audio file types, not just MP3
- Easy to use for Average Joe consumer
- Affordable, preferably without annual subscriptions
- Discover many near-duplicates with the least effort (fast)
- Make few mistakes (false positives)
- Handle lots of files without bugs and crashes (robust)
Before we begin the Clone Olympics, here's a table summarizing the main contenders for the title of Best duplicate audio detective Windows software. Some are specialized for similar music, others can handle also similar video and images. 64-bit programs are capable of handling huge media collections (32-bit programs are limited to 2GB memory)
|
Information correct as of 7 Feb 2023
Typical workflow for finding and removing duplicate files
The workflow of all duplicate file removers more or less comprises 5 basic steps:
- Choose where to search. Define the search scope in certain folders or your entire PC and attached hard disks and other devices like phones. Some duplicate scanners allow access to i-Tunes and other cloud folders.
- Choose what and how to search. We are searching for music files; there are many search options that affect the speed and quality of the results. For this review we concentrate on similar audio content, and the most important parameter is the similarity tolerance, a percent that shows how much similar 2 audio files should be to be considered duplicate. Lowering the similarity threshold will catch more duplicate songs but will also produce false positives, songs that are completely different.
- Wait for results and preview. Audio fingerprint scanning is slow by nature, but you don't need to sit on top of the duplicate scanner while it is searching. Once the results are in, you examine them to make sure the duplicate groups are correct. Most programs offer a music previewer that helps in this quality control phase. If you find any mistakes (false positives) in the results, you can remove the wrong files from further processing.
- Choose what to keep and what to remove. Once the search is finished, duplicate files are grouped together and offer a way to choose what is kept and what gets deleted. Mostly this is done using checkboxes. Manual selection is impractical when many duplicates are found, so programs offer various automatic ways to choose what gets deleted, e.g. the lowest bitrate songs, or the smallest file sizes (poor sound quality)
- Clean up marked items. Finally we want to remove duplicates and free up the hard disk space. This is either done by deleting the duplicate files, or replacing them with hard/symbolic links, which are like file shortcuts that don't take up much space. Some programs allow moving the duplicates in a separate folder, although I don't see the point — other than perhaps as a temporary measure. The goal is freeing up disk space and reducing junk and clutter from our music collection.
Audio Comparer
This duplicate MP3 finder and remover by Bolide software is, according to their website:
...able to work with MP3, MP2, MP1, WMA, AIF, WAV, WavPack, FLAC, APE, AAC, and OGG formats, and actually "listens" to the songs themselves analyzing and recognizing them based on their sounds – just like the human ear. No matter what codec or bitrate your files are saved as, the [ACO] can identify the duplicates.
Audio comparer's user interface is "old school" using a menu and toolbar, but it is fit for purpose — why put lace on a chainsaw? <g>. I expect consumer grade products with a focused job to be self-evident without reading instruction manuals. [ACO] audio groups is not something I understood the purpose of, but there is a comparison wizard to guide the scanning process. The all-important Similarity threshold is set through preferences.
The duplicate music files are shown as a tree, where each node holds a group of similar songs. As you can see from the snapshot above, the whole path of each song \\VBOXSVR\Downloads\... is shown, and that doesn't help seeing the song title itself. Anyway, the results list include relevant audio details like song duration and bitrate. The previewer pane on the top shows more ID3 tags (Artist, Genre, Year etc) and you can also play selected songs.
Many pruning and marking commands are available in the right click (context) menu. Rather than ticking boxes one by one, automarker can automatically tick for removal music files with low or highest quality, depending on your objective (save space or keep the best sound quality albums). Finally click on Delete button to free up the space held by duplicate music files — however this option is disabled in free trial mode.
Verdict
Audio comparer is a decent duplicate music scanner, although its fingerprinting algorithm is slow (see the timing data later on). The management of the duplicate music results is also plain, and you would run in trouble if you had to deal e.g. with 1000s of duplicate songs without some sort of filtering. On the positive side, the GUI is clean, although you can tell that development has stopped since 2011.|
PROS
Discover similar music with missing ID3 tags
Remove duplicates from iTunes collection
|
CONS
Rather slow audio fingerprint algorithm
Not built for large scale duplicate searches
|
Audio DeDupe
This duplicate MP3 finder and deleter by MindGems will, according to their website:
...will find all similar or exact duplicate audio files in a folder and its subfolders. Unlike other duplicate mp3 finders it will "listen" to the music and can recognize a song even if it is saved in different file formats. Set the similarity level to 100% to find exact matches or lower it to find variations and remixes of the same song.
Supports MP3, MP2, MP1, MPA, WAV, OGG, AIFF, AAC, MP4, FLAC, AC3, WavPack (WV), Musepack (MPC) and other music file formats
Audio Dedupe GUI is an improvement over the previous contender. The window shows the list of duplicate music files along with the most important information: the folders being searched, the scan method and similarity threshold, and a preview pane that plays back songs and shows tags for the selection. The only gripe is that you cannot completely hide these panes to see more of the results (why do you want to see the folders searched after the search is over?)
The groups of duplicate music files are presented in alternative background colors (stripes) and have checkboxes to mark files to be removed. Both regular file properties (like size and date modified) and audio-specific details like sample rate and bit resolution are shown. Similarity of duplicates in each group is shown graphically in a separate column, green for 100% (identity), orange for good similarity and red for probably false positive results. As a first time user I don't understand why one item in a group shows as 100% similar where its matching pair is only at 60%? This happens in all groups below 100%, and looks like a bug.
It looks like the program supports filtering of results — although it won't allow it in free trial mode. Autocheck is a feature that automatically marks duplicate items for removal based on audio quality properties or file dates. You can also mark music files manually or with wildcards. Finally you choose the cleanup action, e.g. delete the files and click on Perform button.
Verdict
Audio Dedupe is a good program for finding and deleting similar music files. It's search engine is one of the fastest and makes few mistakes — naturally the error rate increases dramatically when you decrease the similarity threshold from 70 to 60% but that's to be expected. In trial mode it is quite restrictive, it will only show file names of 20 results (although you can sneak-peek the rest using tag information :)|
PROS
Fast audio fingerprinting algorithm
Intuitive user interface
|
CONS
Stability glitches extracting ID3 tags (hangs)
Overly restrictive free trial
|
Duplicate Cleaner Pro
This duplicate cleaner by DigitalVolcano Software can, according to their website:
...can scan and compare audio files using Duplicate Cleaner. Supported formats are MP3, OGG, WMA, M4A, M4P (iTunes), AAC, FLAC and WAV. Compare the same or similar music across different formats.
Duplicate Cleaner will list details of any audio it finds, including length, bit rate and sample rate in addition to the usual tags. The Selection Assistant can help you pick which files to remove - for instance the shortest version or lowest quality.
Duplicate Cleaner has a sleek user interface, which isn't much help in itself, but it also packs a lot of power. It can discover all kinds of duplicate files, not just music (video, pictures and general documents). Managing more kinds of duplicates makes the interface more complicated, but it is well organized in tabs, one selecting the scan type and criteria, one for the folders to scan, one to start the search (progress) and a tab for duplicate results, whenever they are available. The tabs guide you in the steps you must follow to discover duplicate music files.
There are criteria about what to find, perhaps way too many for the average user — who may end up confused, In terms of audio similarity, it can search quickly for tags, and slowly examining the musical content at various lengths of time. Presets allow you to search the first 15 seconds or 2 minutes of each song, but to my experience, the shorter tests didn't find many near-duplicate files, whereas the 2 minute hashes took way too long, beyond the limits of my patience. Oddly enough, despite the wealth of duplicate scan options, I could not find a way to adjust the similarity threshold. The program must be using a fixed, conservative similarity setting internally, and as a result it finds far fewer duplicate songs among the competitors in this mini-olympiad.
The duplicate songs discovered are presented grouped in striped background colors, and use checkboxes for marking unwanted songs to be removed. Several audio properties are listed with the files (Duration, Album, Title, Artist etc) — but no similarity metric is shown. The embedded media previewer comes handy to compare the songs in each duplicate group — I saw some intermittent playback errors but couldn't find a pattern to the fault. The conservative (high) similarity level used means that there are usually no errors or false duplicates — but again note that fewer similar music are discovered. You gain some, you lose some...
You can filter the duplicate files found, by any visible property, e.g. filename or artist, which comes handy managing large numbers of results. The selection assistant looks powerful, and can mark with many prioritised rules at a time (e.g. shortest duration followed by oldest date modified). For cleanup, you get the full range of possibilities, deleting the duplicates, creating hard and symbolic links, and exporting the results as CSV text.
Verdict
Duplicate Cleaner Pro is a mature program for finding and removing duplicate files of all sorts, including similar audio and video. It has the most polished UI in this review, which assists the workflow. For music (and video) duplicates it is let down by its slow hash fingerprint creation algorithm. The program will store hashes in a cache, but there won't be any use of repeated scans, if one cannot complete the initial one to begin with!|
PROS
Finds all kinds of duplicates, value for money
Very pretty to look at
|
CONS
Relatively slow audio hash creation
Conservative in similarity tolerance, finds less duplicates
|
Duplicate Media Finder
Duplicate Media Finder (finds and deletes duplicate files) will, according to their website:
Just like for image or video files, Duplicate Media Finder will analyse the content of audio files. It can identify similar music among several formats (mp3, ogg, oga, ac3, flac, m4a(aac & alac), mp2, wav, wma, opus, wv, amr).
...you can automatically select duplicates to delete according to certain criteria, such as sound quality or sampling rate, but also file size or date…
This is another program that finds not only music but also similar video and images. As you can see from the pic above, the UI tries to be "cool" but IMHO they need to hire a stylist — anyway I personally always go for function over form. However the oddness of the UI also hampers usability at least until you realise that some buttons are really toggle switches, e.g. to select what to search for — they go partially green when active.
There are kind-of-tabs for search setup (where and what to search). There are few options offered for tweaking, but they are the most useful ones, so no complaint there. For our review the required options are search for similar media, and all the other criteria aren't required (they are well hidden from view). A sensitivity trackbar adjusts the similarity tolerance. You select folders to scan using a checkbox-enabled folder tree, then click Start button and wait for the results.
The duplicate music files found tab is eccentric, showing thumbnails organized in rows, each row holding a set of duplicate songs. Perhaps this display mode is useful for image thumbnails, but for audio it is wasting space and hiding information about the duplicates involved. The first column is the master file, but it's not clear what criteria are used for the automatic choice. Managing results is clunky, done through the context menu. A glaring omission is the lack of checkboxes to permanently mark duplicates. Selecting items with the mouse is fragile, one wrong click and you lose all the selection! And are there no means for filtering the results? Where is the Delete button? Are we supposed to delete duplicate files one by one with the context menu?
It is a pity really, because the this program's duplicates search engine is by far the fastest in this review (for music; the performance for videos isn't stellar), and does no mistakes. Apparently they make use of all the available cores and processors, and it shows. It would probably be even faster if they included a 64-bit version. Its free trial mode is frugal, but with this kind of tool that's used infrequently, I can't blame them for being over-protective.
Verdict
If Duplicate Media Finder was a movie, I would have thought that they ran out of money midway through the production. It has an excellent part for discovering similar media files, but it is almost unusable as a cleaner of duplicates, you may as well manually delete duplicates one by one. And nobody has time for such software, sorry. Perhaps their main audience is people that deal with pictures and thumbnails?|
PROS
Finds duplicate music, video and images
Very fast and reliable fingerprint algorithm
|
CONS
Quirky interface
Poor duplicate file management
|
Duplicate Music Fixer
This Duplicate music file finder and fixer by Systweak can, according to their website:
...can find redundant music files that take unnecessary storage space. Quickly find, remove and organize your music collection.
Duplicate Music Fixer finds and removes duplicate music files giving you a clutter free music collection. It detects similarities in MP3 files and helps retain highest quality audio files automatically.
Duplicate Music Fixer is an extremely focused program that offers very few adjustments to the user, therefore it is very easy to use. The tabbed interface guides your steps very quickly. You only choose the folders to search, including the local music library and iTunes (eh? we're Windows users here :). No setting of quick/slow scan, it just searches for similar music with its own preset and unknown settings. But it works, albeit slowly.
Switch to the next tab for the results, and you find the similar music files organized in bands, clearly. Checkboxes mark the songs to remove, either manually or using the selection assistant. The automatic selection of duplicates favors keeping higher quality (bitrate) songs, and there's little room for adjustment. If you don't like the default serving, you must mark everything manually. As for variety in cleanup actions, you only get a delete button, and that's that. Creating symbolic links is for only rocket scientists anyway <g>
Verdict
If you have a small music library, and don't need many frills, Duplicate Music Fixer can do the job, finding both identical and similarly-sounding songs to get rid of. But it's very slow searching and limited (no filtering!) managing the results for any kind of substantial use.|
PROS
Very easy to use for duplicate music
Find similar songs searching the content
|
CONS
Extremely slow fingerprinting algorithm
Restricted flexibility for search and cleanup
|
i-DeClone
DOWNLOAD FREE TRIAL
i-DeClone is our own product that cleans-up duplicate and similar files, including music. According to the website it:
...can recover the space wasted by identical files stored many times, or similar versions of the same information (e.g. similar photos), using advanced AI algorithms. The scan process is accurate, reliable and fast
i-DeClone is industrial-strength, built to handle thousands of duplicate files and folders
i-DeClone began its life as a tool inside our xplorer², so it feels almost as a file manager. Its user interface is spartan but easy to use, because it hides its complexities well, while at the same time allowing expert users to go deeper if they so desire. It is a generic duplicate file management tool, that finds duplicate and similar music, videos, office documents, photos, e-books and even whole folders.
The UI comprises just a toolbar, having buttons dealing with the main steps of the workflow. Some split-buttons show a drop-down menu when clicked on the arrow. Click on Start scan to begin a new search, and set the type to Music, add some folders to scan, then click on find similar files — that's all the information required. i-DeClone shows a summary of what is about to happen and guides you all the way to marking the duplicate audio files discovered.
Similar audio results are shown in a list like windows explorer, each set of similar files in its own group. i-DeClone shows relevant ID3 tags and the similarity % of each group (high numbers mean good matches). Right click on the column header to choose additional file or audio properties to see (300+ windows system properties are available). There's an embedded windows media player to listen to the songs, ensuring they are correctly matched.
Use the checkboxes or Mark wizard to mark duplicate files for removal, using date or quality criteria. The quickfilter box on the toolbar can filter the results by text appearing in any field, be it filename, path or bitrate. i-DeClone can manage thousands of duplicates in this way. For Clean-up you can opt for deletion to save space, or create hard or symbolic links. You can even save the results and work with them later.
i-DeClone lets you revise a duplicate files scan, e.g. using a different similarity threshold on the same audio files. So if you happen to find too few hits, or too many false duplicates, you can quickly amend the search, using the files already examined and cached in memory.
Verdict
i-DeClone packs lots of AI-driven power to discover any kind of duplicate files, be it music or office documents. The interface is simple and explorer-like but very ergonomic. As you will see in timing data section it discovers many near-duplicate songs with good performance in terms of CPU effort.|
PROS
Complete solution for all duplicate file types
Discover many similar songs in good time
|
CONS
Some of its results are false duplicates
It could use a GUI facelift
|
Similarity
Similarity by GAR Software will, according to their website:
...rapidly scans your music collection and shows all duplicate music files you may have. The comparison powered by "acoustic fingerprint" technology considers the actual contents of files, not just tags or filenames, and thus ensures the extreme accuracy of similarity estimation. Moreover, the comparison isn’t dependent of music file format; the application supports almost every file format in full (MP3, AAC, M4A, WMA, OGG, WAV, FLAC, APE, WV, MPC, OPUS, TTA and others through codecs)
As the name implies, Similarity excels at finding similar music files of various formats and encodings. By the looks of it, Similarity was developed as an audio quality control and analysis tool, and removing duplicate songs was an afterthought, tacked on rather hastily. I mean, if you need to use javascript to mark duplicate files, then surely this isn't meant to be used by common people, is it?
To start a scan, use the options toolbar button to set the similarity thresholds. The similarity metric is a combination of musical content (spectral fingerprint I presume), audio ID3 tags and an unspecified precise algorithm, which is professed as "fast and accurate" but is sadly absent in free trial mode. Then add a few folders to scan and click on play button to begin the search. The algorithm must be analyzing the entire song, because it is very slow. But wait a while, have a coffee and the similarity results will start arriving.
As seen in the window snapshot above, similar songs are grouped in alternating background color stripes, as is typical in this type of software. There are also some tree-like arrows that expand each line, but it is not clear what is the point of this. Anyway the usual file and ID3 tag details are available in the list. If you sort by content similarity, then the closest matches (near 100%) are towards the end of the list. So far, so good.
And then it hits you... where are the checkboxes? To mark duplicates manually you need to use the context menu (or press <SPACE> key), which merely turns the row red — it took me some time to realize this faintest of UI changes. If you are familiar with javascript (!) you can write your own mini-program to select duplicates "automatically". Deleting marked songs is available through the context menu as well — no further programming required!
Verdict
Similarity is a robust and impressive audio analysis tool, generating spectrums and sonograms, meant to weed out music files with low quality sound. Removal of similar songs is something Similarity does on the side, and isn't fit for the average consumer. If they ever want to target the duplicate file removal market, it won't be too hard to modify the program, as they already have a powerful audio analysis engine.|
PROS
Impressive audio analysis and quality control
Audio similarity by combined content/tag metric
|
CONS
Slow analysis of entire songs
No filters or automatic duplicate marking
|
Wise duplicate finder
Wise duplicate finder and cleaner by WiseCleaner software is, according to their website:
...a duplicate file management tool that can help you find and delete duplicate files by comparing file name, file size or contents. Get rid of annoying duplicates and free up more space on your hard disks.
You can choose any type of duplicates to manage, for example find duplicate photos, find duplicate videos, find duplicate documents, etc.
This software isn't in the same league as the other duplicate scanners in this review, it is included to demonstrate that you need specialized tools for audio similarity — a task Wise duplicate finder falls well short of.
The ribbon based interface is quite simple. You select the folders to scan and the comparison mode from Home tab, set the file type to Audio files then click on Scan to begin the search. I set the mode to partial match that apparently allows slight differences in filenames and dates (fuzzy matching).
Very quickly you see the duplicate (identical) files showing up as in the pic above, using stripes for each duplicate group, and checkboxes for marking songs for deletion. There is a preview pane that plays songs, but there's no sight of music-related ID3 tags or attributes. You can automatically select older or newer files to remove using date modified or created, and delete the selected items to free up wasted disk space.
There's nothing wrong with this kind of software, it will quickly locate identical files of any type (not just music files), As you will shortly see in the experimental results part of this review, it only finds 32 out of the ~200 musically identical songs in the test environment.
|
PROS
Cheap and cheerful
Finds identical files quickly
|
CONS
Cannot find similar music files
Requires elevation to run (!?)
|
Duplicate music scanner performance results
We have reached the essence of this comparative review, performance data on the job of actually discovering duplicate music files. Which program out of the above 8 will find the most duplicates without making too many mistakes, and in reasonable time? All duplicate finders were tested on the same set of 3814 songs a mixture of mostly MP3 and a few FLAC, M4A, WMA and WAV, a real life music collection totalling 35.3GB in size. Duplicate songs hide in the guise of different encoding formats (MP3 vs WMA), different bitrates (sampling quality), and slightly different lengths (on account of variable silence at the beginning of a song).
Between them, these programs found 200 duplicate songs (32 of which were 100% binary identical files); the most similar songs were discovered by everybody, whereas some borderline-similar songs (e.g. alternate takes and bonus tracks) were discovered only by a few programs (and each program found a different set of those songs, according to the acoustic fingerprint used). None of the reviewed software got all of the duplicates right, but some fared better than the others.
Default "out of the box" properties were used for all duplicate finder software, unless they didn't work properly (see the notes section). I always picked a scan option that examined the content for similarity, using audio signatures, at default similarity tolerance levels. Here are the timing results of the comparative test: (best results highlighted in green and worst in red)
|
[1] using audio match setting compare first 15 seconds; any other option was prohibitively slow
[2] experiment performed on smaller set of files then extrapolated, due to extremely slow algorithm
[3] the error rate can be eliminated by increasing the similarity threshold slightly, with a slight reduction in duplicates discovered
[4] this program can only identify identical files, so it isn't a contender in this review
[5] throughput combination metric favors engines that find the most duplicates in the shortest space of time
As you can see from these results, when heavy computational algorithms execute, what matters isn't how the program looks but how efficiently it performs. We are not shopping for oil paintings to hang on the wall, but for heavy duty tools. As expected, the "naive" byte checking Wise duplicate finder [WDF] found only 32 identical music files, albeit quickly — and that's all I'm going to say about it.
There is no clear cut winner that excels in every department. i-DeClone [IDE] found the most duplicates (180) but also made the most mistakes (24). Duplicate media finder [DMF] was the fastest but found 30% less duplicates (132). Similarity [SIM] found almost as many similar songs as i-DeClone, but it took 6 times longer to find them (23m 59 sec). Which one would you pick?
Processing speed is only half the story, one must also account for ease of management of duplicates discovered, and we saw that the speed king [DMF] is not very good when it comes to marking and deleting duplicate music files. Arguably the most powerful tool for managing duplicate groups is Duplicate cleaner [DCL], but what good is that when it finds the fewest (70) duplicate music files to begin with?
🏆 i-DeClone by Zabkat LTD
So the title for the best duplicate audio tool goes to the in-house product i-DeClone, how predictable!? But wait, this is not a marketing scam, this acclaim is backed by reproducible evidence. i-DeClone has the second fastest duplicate search engine throughput (42.19 duplicates per minute) and finds the most similar songs, and it is very capable dealing with previewing, validating and ultimately getting rid of the duplicates discovered. And unlike other specialized tools in this review, it deals with all types of media and documents, not just audio.
Please download the free trial and see for yourself, it works full throttle and it will let you remove up to 100 duplicates free of charge. Thank you!
