Review: the best duplicate/similar song finder and cleaner software

  How do you discover near-duplicate music?

Take for instance the following 2 music files. It is the same song, but the file size of one is almost double (320kbps bitrate), the duration of one is 4 seconds longer (probably on account of silence at start) and the ID3 tags are all over the place — still they represent the same song!

Name					Size	Genre		Length		Bit rate	Year
02-Siva.mp3				6.01MB	Alternative	00:04:25	190kbps		1991
The Smashing Pumpkins - Siva.mp3	10.0MB	Mp3		00:04:21	320kbps		2001

What we need therefore is software that can find musical duplicates, allowing for differences in amplitude and recording quality and file formats (MP3,WMA,AIF,OGG,FLAC,AAC,APE...). Media files need a computer program that can understand the music, not the raw bytes stored in them. The bytes may differ, even the waveform itself slightly, but musically-aware software will detect the similarity, almost as a human listener would.

Most of the aforementioned 102 duplicate cleaners are not "musically savvy". They either rely on naive byte-comparison, or at best use ID3 tags like song title, artist and composer for comparing music files — but these tags are notoriously unreliable for downloaded music. I have narrowed it down to just seven (7) computer programs (for Windows 7/10/11) that analyze the music for similarity and will find musical duplicates, which in alphabetical order are:

• Audio comparer by Bolide software [ACO]
• Audio dedupe by MindGems [ADE]
• Duplicate cleaner Pro by DigitalVolcano [DCL]
• Duplicate Media Finder [DMF]
• Duplicate music fixer by Systweak [DFI]
• i-DeClone by Zabkat [IDE]
• Similarity by GAR Software [SIM]

I will also consider another popular program called Wise duplicate finder, just to show you how "naive" duplicate finders are not up to the task of discovering musically duplicate songs. All these programs are shareware with free trial, so you would expect a decent quality in departments such as:

• Handle many audio file types, not just MP3
• Easy to use for Average Joe consumer
• Affordable, preferably without annual subscriptions
• Discover many near-duplicates with the least effort (fast)
• Make few mistakes (false positives)
• Handle lots of files without bugs and crashes (robust)

Before we begin the Clone Olympics, here's a table summarizing the main contenders for the title of Best duplicate audio detective Windows software. Some are specialized for similar music, others can handle also similar video and images. 64-bit programs are capable of handling huge media collections (32-bit programs are limited to 2GB memory)

Name/Website Version   Price   (US $) Download   size (MB) 64 bit   Audio   Video   Images Notes     ■ Audio comparer [ACO] 1.7 $35 2.84 — — — Last release in 2011   ■ Audio dedupe [ADE] 4.9 $40 7.6 — — separate programs sold for images etc   ■ Duplicate cleaner Pro [DCL] 5.18 $40 62.2   ■ Duplicate Media Finder [DMF] 9.01 $46 26.6 —   ■ Duplicate music fixer [DFI] 2.1 $30 8.8 — — — some virustotal issues   ■ i-DeClone [IDE] 1.52 $30 2.3   ■ Similarity [SIM] 2.5 $20 13.1 —   ■ Wise Duplicate Finder [WDF] 2.0 $15 9.4 scans bytes, not music

Typical workflow for finding and removing duplicate files

The workflow of all duplicate file removers more or less comprises 5 basic steps:

1. Choose where to search. Define the search scope in certain folders or your entire PC and attached hard disks and other devices like phones. Some duplicate scanners allow access to i-Tunes and other cloud folders.
    
2. Choose what and how to search. We are searching for music files; there are many search options that affect the speed and quality of the results. For this review we concentrate on similar audio content, and the most important parameter is the similarity tolerance, a percent that shows how much similar 2 audio files should be to be considered duplicate. Lowering the similarity threshold will catch more duplicate songs but will also produce false positives, songs that are completely different.
    
3. Wait for results and preview. Audio fingerprint scanning is slow by nature, but you don't need to sit on top of the duplicate scanner while it is searching. Once the results are in, you examine them to make sure the duplicate groups are correct. Most programs offer a music previewer that helps in this quality control phase. If you find any mistakes (false positives) in the results, you can remove the wrong files from further processing.
    
4. Choose what to keep and what to remove. Once the search is finished, duplicate files are grouped together and offer a way to choose what is kept and what gets deleted. Mostly this is done using checkboxes. Manual selection is impractical when many duplicates are found, so programs offer various automatic ways to choose what gets deleted, e.g. the lowest bitrate songs, or the smallest file sizes (poor sound quality)
    
5. Clean up marked items. Finally we want to remove duplicates and free up the hard disk space. This is either done by deleting the duplicate files, or replacing them with hard/symbolic links, which are like file shortcuts that don't take up much space. Some programs allow moving the duplicates in a separate folder, although I don't see the point — other than perhaps as a temporary measure. The goal is freeing up disk space and reducing junk and clutter from our music collection.
    

Audio Comparer

...able to work with MP3, MP2, MP1, WMA, AIF, WAV, WavPack, FLAC, APE, AAC, and OGG formats, and actually "listens" to the songs themselves analyzing and recognizing them based on their sounds – just like the human ear. No matter what codec or bitrate your files are saved as, the [ACO] can identify the duplicates.

Audio comparer's user interface is "old school" using a menu and toolbar, but it is fit for purpose — why put lace on a chainsaw? <g>. I expect consumer grade products with a focused job to be self-evident without reading instruction manuals. [ACO] audio groups is not something I understood the purpose of, but there is a comparison wizard to guide the scanning process. The all-important Similarity threshold is set through preferences.

The duplicate music files are shown as a tree, where each node holds a group of similar songs. As you can see from the snapshot above, the whole path of each song \\VBOXSVR\Downloads\... is shown, and that doesn't help seeing the song title itself. Anyway, the results list include relevant audio details like song duration and bitrate. The previewer pane on the top shows more ID3 tags (Artist, Genre, Year etc) and you can also play selected songs.

Many pruning and marking commands are available in the right click (context) menu. Rather than ticking boxes one by one, automarker can automatically tick for removal music files with low or highest quality, depending on your objective (save space or keep the best sound quality albums). Finally click on Delete button to free up the space held by duplicate music files — however this option is disabled in free trial mode.

Verdict

PROS Discover similar music with missing ID3 tags Remove duplicates from iTunes collection

CONS Rather slow audio fingerprint algorithm Not built for large scale duplicate searches

Audio DeDupe

...will find all similar or exact duplicate audio files in a folder and its subfolders. Unlike other duplicate mp3 finders it will "listen" to the music and can recognize a song even if it is saved in different file formats. Set the similarity level to 100% to find exact matches or lower it to find variations and remixes of the same song.
Supports MP3, MP2, MP1, MPA, WAV, OGG, AIFF, AAC, MP4, FLAC, AC3, WavPack (WV), Musepack (MPC) and other music file formats

Audio Dedupe GUI is an improvement over the previous contender. The window shows the list of duplicate music files along with the most important information: the folders being searched, the scan method and similarity threshold, and a preview pane that plays back songs and shows tags for the selection. The only gripe is that you cannot completely hide these panes to see more of the results (why do you want to see the folders searched after the search is over?)

The groups of duplicate music files are presented in alternative background colors (stripes) and have checkboxes to mark files to be removed. Both regular file properties (like size and date modified) and audio-specific details like sample rate and bit resolution are shown. Similarity of duplicates in each group is shown graphically in a separate column, green for 100% (identity), orange for good similarity and red for probably false positive results. As a first time user I don't understand why one item in a group shows as 100% similar where its matching pair is only at 60%? This happens in all groups below 100%, and looks like a bug.

It looks like the program supports filtering of results — although it won't allow it in free trial mode. Autocheck is a feature that automatically marks duplicate items for removal based on audio quality properties or file dates. You can also mark music files manually or with wildcards. Finally you choose the cleanup action, e.g. delete the files and click on Perform button.

Verdict

PROS Fast audio fingerprinting algorithm Intuitive user interface

CONS Stability glitches extracting ID3 tags (hangs) Overly restrictive free trial

Duplicate Cleaner Pro

...can scan and compare audio files using Duplicate Cleaner. Supported formats are MP3, OGG, WMA, M4A, M4P (iTunes), AAC, FLAC and WAV. Compare the same or similar music across different formats.
Duplicate Cleaner will list details of any audio it finds, including length, bit rate and sample rate in addition to the usual tags. The Selection Assistant can help you pick which files to remove - for instance the shortest version or lowest quality.

Duplicate Cleaner has a sleek user interface, which isn't much help in itself, but it also packs a lot of power. It can discover all kinds of duplicate files, not just music (video, pictures and general documents). Managing more kinds of duplicates makes the interface more complicated, but it is well organized in tabs, one selecting the scan type and criteria, one for the folders to scan, one to start the search (progress) and a tab for duplicate results, whenever they are available. The tabs guide you in the steps you must follow to discover duplicate music files.

There are criteria about what to find, perhaps way too many for the average user — who may end up confused, In terms of audio similarity, it can search quickly for tags, and slowly examining the musical content at various lengths of time. Presets allow you to search the first 15 seconds or 2 minutes of each song, but to my experience, the shorter tests didn't find many near-duplicate files, whereas the 2 minute hashes took way too long, beyond the limits of my patience. Oddly enough, despite the wealth of duplicate scan options, I could not find a way to adjust the similarity threshold. The program must be using a fixed, conservative similarity setting internally, and as a result it finds far fewer duplicate songs among the competitors in this mini-olympiad.

The duplicate songs discovered are presented grouped in striped background colors, and use checkboxes for marking unwanted songs to be removed. Several audio properties are listed with the files (Duration, Album, Title, Artist etc) — but no similarity metric is shown. The embedded media previewer comes handy to compare the songs in each duplicate group — I saw some intermittent playback errors but couldn't find a pattern to the fault. The conservative (high) similarity level used means that there are usually no errors or false duplicates — but again note that fewer similar music are discovered. You gain some, you lose some...

You can filter the duplicate files found, by any visible property, e.g. filename or artist, which comes handy managing large numbers of results. The selection assistant looks powerful, and can mark with many prioritised rules at a time (e.g. shortest duration followed by oldest date modified). For cleanup, you get the full range of possibilities, deleting the duplicates, creating hard and symbolic links, and exporting the results as CSV text.

Verdict

PROS Finds all kinds of duplicates, value for money Very pretty to look at

CONS Relatively slow audio hash creation Conservative in similarity tolerance, finds less duplicates

Duplicate Media Finder

Just like for image or video files, Duplicate Media Finder will analyse the content of audio files. It can identify similar music among several formats (mp3, ogg, oga, ac3, flac, m4a(aac & alac), mp2, wav, wma, opus, wv, amr).
...you can automatically select duplicates to delete according to certain criteria, such as sound quality or sampling rate, but also file size or date…

This is another program that finds not only music but also similar video and images. As you can see from the pic above, the UI tries to be "cool" but IMHO they need to hire a stylist — anyway I personally always go for function over form. However the oddness of the UI also hampers usability at least until you realise that some buttons are really toggle switches, e.g. to select what to search for — they go partially green when active.

There are kind-of-tabs for search setup (where and what to search). There are few options offered for tweaking, but they are the most useful ones, so no complaint there. For our review the required options are search for similar media, and all the other criteria aren't required (they are well hidden from view). A sensitivity trackbar adjusts the similarity tolerance. You select folders to scan using a checkbox-enabled folder tree, then click Start button and wait for the results.

The duplicate music files found tab is eccentric, showing thumbnails organized in rows, each row holding a set of duplicate songs. Perhaps this display mode is useful for image thumbnails, but for audio it is wasting space and hiding information about the duplicates involved. The first column is the master file, but it's not clear what criteria are used for the automatic choice. Managing results is clunky, done through the context menu. A glaring omission is the lack of checkboxes to permanently mark duplicates. Selecting items with the mouse is fragile, one wrong click and you lose all the selection! And are there no means for filtering the results? Where is the Delete button? Are we supposed to delete duplicate files one by one with the context menu?

It is a pity really, because the this program's duplicates search engine is by far the fastest in this review (for music; the performance for videos isn't stellar), and does no mistakes. Apparently they make use of all the available cores and processors, and it shows. It would probably be even faster if they included a 64-bit version. Its free trial mode is frugal, but with this kind of tool that's used infrequently, I can't blame them for being over-protective.

Verdict

PROS Finds duplicate music, video and images Very fast and reliable fingerprint algorithm

CONS Quirky interface Poor duplicate file management

Duplicate Music Fixer

...can find redundant music files that take unnecessary storage space. Quickly find, remove and organize your music collection.
Duplicate Music Fixer finds and removes duplicate music files giving you a clutter free music collection. It detects similarities in MP3 files and helps retain highest quality audio files automatically.

Duplicate Music Fixer is an extremely focused program that offers very few adjustments to the user, therefore it is very easy to use. The tabbed interface guides your steps very quickly. You only choose the folders to search, including the local music library and iTunes (eh? we're Windows users here :). No setting of quick/slow scan, it just searches for similar music with its own preset and unknown settings. But it works, albeit slowly.

Switch to the next tab for the results, and you find the similar music files organized in bands, clearly. Checkboxes mark the songs to remove, either manually or using the selection assistant. The automatic selection of duplicates favors keeping higher quality (bitrate) songs, and there's little room for adjustment. If you don't like the default serving, you must mark everything manually. As for variety in cleanup actions, you only get a delete button, and that's that. Creating symbolic links is for only rocket scientists anyway <g>

Verdict

PROS Very easy to use for duplicate music Find similar songs searching the content

CONS Extremely slow fingerprinting algorithm Restricted flexibility for search and cleanup

i-DeClone

DOWNLOAD FREE TRIAL

i-DeClone is our own product that cleans-up duplicate and similar files, including music. According to the website it:

...can recover the space wasted by identical files stored many times, or similar versions of the same information (e.g. similar photos), using advanced AI algorithms. The scan process is accurate, reliable and fast
i-DeClone is industrial-strength, built to handle thousands of duplicate files and folders

i-DeClone began its life as a tool inside our xplorer², so it feels almost as a file manager. Its user interface is spartan but easy to use, because it hides its complexities well, while at the same time allowing expert users to go deeper if they so desire. It is a generic duplicate file management tool, that finds duplicate and similar music, videos, office documents, photos, e-books and even whole folders.

The UI comprises just a toolbar, having buttons dealing with the main steps of the workflow. Some split-buttons show a drop-down menu when clicked on the arrow. Click on Start scan to begin a new search, and set the type to Music, add some folders to scan, then click on find similar files — that's all the information required. i-DeClone shows a summary of what is about to happen and guides you all the way to marking the duplicate audio files discovered.

Similar audio results are shown in a list like windows explorer, each set of similar files in its own group. i-DeClone shows relevant ID3 tags and the similarity % of each group (high numbers mean good matches). Right click on the column header to choose additional file or audio properties to see (300+ windows system properties are available). There's an embedded windows media player to listen to the songs, ensuring they are correctly matched.

Use the checkboxes or Mark wizard to mark duplicate files for removal, using date or quality criteria. The quickfilter box on the toolbar can filter the results by text appearing in any field, be it filename, path or bitrate. i-DeClone can manage thousands of duplicates in this way. For Clean-up you can opt for deletion to save space, or create hard or symbolic links. You can even save the results and work with them later.

i-DeClone lets you revise a duplicate files scan, e.g. using a different similarity threshold on the same audio files. So if you happen to find too few hits, or too many false duplicates, you can quickly amend the search, using the files already examined and cached in memory.

Verdict

PROS Complete solution for all duplicate file types Discover many similar songs in good time

CONS Some of its results are false duplicates It could use a GUI facelift

Similarity

...rapidly scans your music collection and shows all duplicate music files you may have. The comparison powered by "acoustic fingerprint" technology considers the actual contents of files, not just tags or filenames, and thus ensures the extreme accuracy of similarity estimation. Moreover, the comparison isn’t dependent of music file format; the application supports almost every file format in full (MP3, AAC, M4A, WMA, OGG, WAV, FLAC, APE, WV, MPC, OPUS, TTA and others through codecs)

As the name implies, Similarity excels at finding similar music files of various formats and encodings. By the looks of it, Similarity was developed as an audio quality control and analysis tool, and removing duplicate songs was an afterthought, tacked on rather hastily. I mean, if you need to use javascript to mark duplicate files, then surely this isn't meant to be used by common people, is it?

To start a scan, use the options toolbar button to set the similarity thresholds. The similarity metric is a combination of musical content (spectral fingerprint I presume), audio ID3 tags and an unspecified precise algorithm, which is professed as "fast and accurate" but is sadly absent in free trial mode. Then add a few folders to scan and click on play button to begin the search. The algorithm must be analyzing the entire song, because it is very slow. But wait a while, have a coffee and the similarity results will start arriving.

As seen in the window snapshot above, similar songs are grouped in alternating background color stripes, as is typical in this type of software. There are also some tree-like arrows that expand each line, but it is not clear what is the point of this. Anyway the usual file and ID3 tag details are available in the list. If you sort by content similarity, then the closest matches (near 100%) are towards the end of the list. So far, so good.

And then it hits you... where are the checkboxes? To mark duplicates manually you need to use the context menu (or press <SPACE> key), which merely turns the row red — it took me some time to realize this faintest of UI changes. If you are familiar with javascript (!) you can write your own mini-program to select duplicates "automatically". Deleting marked songs is available through the context menu as well — no further programming required!

Verdict

PROS Impressive audio analysis and quality control Audio similarity by combined content/tag metric

CONS Slow analysis of entire songs No filters or automatic duplicate marking

Wise duplicate finder

...a duplicate file management tool that can help you find and delete duplicate files by comparing file name, file size or contents. Get rid of annoying duplicates and free up more space on your hard disks.
You can choose any type of duplicates to manage, for example find duplicate photos, find duplicate videos, find duplicate documents, etc.

This software isn't in the same league as the other duplicate scanners in this review, it is included to demonstrate that you need specialized tools for audio similarity — a task Wise duplicate finder falls well short of.

The ribbon based interface is quite simple. You select the folders to scan and the comparison mode from Home tab, set the file type to Audio files then click on Scan to begin the search. I set the mode to partial match that apparently allows slight differences in filenames and dates (fuzzy matching).

Very quickly you see the duplicate (identical) files showing up as in the pic above, using stripes for each duplicate group, and checkboxes for marking songs for deletion. There is a preview pane that plays songs, but there's no sight of music-related ID3 tags or attributes. You can automatically select older or newer files to remove using date modified or created, and delete the selected items to free up wasted disk space.

There's nothing wrong with this kind of software, it will quickly locate identical files of any type (not just music files), As you will shortly see in the experimental results part of this review, it only finds 32 out of the ~200 musically identical songs in the test environment.

PROS Cheap and cheerful Finds identical files quickly

CONS Cannot find similar music files Requires elevation to run (!?)

Duplicate music scanner performance results

Between them, these programs found 200 duplicate songs (32 of which were 100% binary identical files); the most similar songs were discovered by everybody, whereas some borderline-similar songs (e.g. alternate takes and bonus tracks) were discovered only by a few programs (and each program found a different set of those songs, according to the acoustic fingerprint used). None of the reviewed software got all of the duplicates right, but some fared better than the others.

Default "out of the box" properties were used for all duplicate finder software, unless they didn't work properly (see the notes section). I always picked a scan option that examined the content for similarity, using audio signatures, at default similarity tolerance levels. Here are the timing results of the comparative test: (best results highlighted in green and worst in red)

[ACO]     [ADE]     [DCL]     [DMF]     [DFI]     [IDE]     [SIM]        [WDF]     ■ Duplicate songs discovered 94 113 70 132 120 180 176 32   ■ False positives (mistakes) 0 6 0 0 6 24 14 0   ■ Total scan time in minutes (MM:SS)   26:20 3:15 6:29 1:51 64:44 4:16 23:59 0:03   ■ Throughput rate (duplicates/min) [5] 3.57 34.77 10.80 71.35 1.85 42.19 7.34 —   ■ Notes [1] [2] [3] [4]

[1] using audio match setting compare first 15 seconds; any other option was prohibitively slow
[2] experiment performed on smaller set of files then extrapolated, due to extremely slow algorithm
[3] the error rate can be eliminated by increasing the similarity threshold slightly, with a slight reduction in duplicates discovered
[4] this program can only identify identical files, so it isn't a contender in this review
[5] throughput combination metric favors engines that find the most duplicates in the shortest space of time

As you can see from these results, when heavy computational algorithms execute, what matters isn't how the program looks but how efficiently it performs. We are not shopping for oil paintings to hang on the wall, but for heavy duty tools. As expected, the "naive" byte checking Wise duplicate finder [WDF] found only 32 identical music files, albeit quickly — and that's all I'm going to say about it.

There is no clear cut winner that excels in every department. i-DeClone [IDE] found the most duplicates (180) but also made the most mistakes (24). Duplicate media finder [DMF] was the fastest but found 30% less duplicates (132). Similarity [SIM] found almost as many similar songs as i-DeClone, but it took 6 times longer to find them (23m 59 sec). Which one would you pick?

Processing speed is only half the story, one must also account for ease of management of duplicates discovered, and we saw that the speed king [DMF] is not very good when it comes to marking and deleting duplicate music files. Arguably the most powerful tool for managing duplicate groups is Duplicate cleaner [DCL], but what good is that when it finds the fewest (70) duplicate music files to begin with?

So the title for the best duplicate audio tool goes to the in-house product i-DeClone, how predictable!? But wait, this is not a marketing scam, this acclaim is backed by reproducible evidence. i-DeClone has the second fastest duplicate search engine throughput (42.19 duplicates per minute) and finds the most similar songs, and it is very capable dealing with previewing, validating and ultimately getting rid of the duplicates discovered. And unlike other specialized tools in this review, it deals with all types of media and documents, not just audio.

Please download the free trial and see for yourself, it works full throttle and it will let you remove up to 100 duplicates free of charge. Thank you!