<p>It might be overkill in your situation since the videos aren't actually different resolutions, frame rates and or codecs, but still :</p>
<p>As with another answer <a href="https://video.stackexchange.com/a/33521/32436">https://video.stackexchange.com/a/33521/32436</a> :</p>
<blockquote>
<p>You could check out <a href="https://theophanemayaud.github.io/video-simili-duplicate-cleaner/" rel="nofollow noreferrer">https://theophanemayaud.github.io/video-simili-duplicate-cleaner/</a> (macOS and Windows binaries available) It's open source, based on vidupe and QT and relies on ffmpeg.</p>
<p>The idea is to extract a few (eg 2, the first and the last) frames, and compare them. If the duration is close, and the frames look similar : you've got a duplicate.</p>
<p>It uses opencv for frame comparison, and ffmpeg for frame extraction (ffmpeg is also for video metadata which is very useful when trying to tell which video to keep !). Since it's open source, you can improve it to fit your needs !</p>
</blockquote>
<p>Because it looks at the video data, you shouldn't have problems of tags or other metadata that has changed !</p>

CC BY-SA 4.0
2021-03-25T14:06:09.037Z
2022-03-16T13:40:33.627Z
2022-03-16T13:40:33.627Z
Théophane
<p>I had the same question and I managed to answer it by creating an open source program around ffmpeg and findimagedupes. Feel free to try it :
<a href="https://github.com/PierreCrette/VideoDedup/blob/master/README.md" rel="nofollow noreferrer">https://github.com/PierreCrette/VideoDedup/blob/master/README.md</a></p>

CC BY-SA 4.0
2018-11-25T11:16:53.577Z
2018-11-25T11:16:53.577Z
Pierre
<p>I am looking for an automated method to identify duplicated video files from my large collection of videos.</p>

<p>I have used fslint to do this in the past, but I started fixing the EXIF tags on my videos, which makes fslint identify them as different files.  The only difference between the original file and the modified files are the EXIF tags, which in most cases have simply been time-shifted to correct for timezone or camera offset.  The videos have not had any encoding or conversion of any type.</p>

<p>Is there a tool something that I can use to identify duplicate videos based only on the AV streams, i.e. ignoring the tags?  I'm essentially looking to duplicate fslint-like functionality, but ignoring the EXIF tags.  Does such a tool exist?  </p>

<p>I have a suspicion that ffmpeg can be used to do this, but all I've been able to find so far is <a href="https://stackoverflow.com/questions/25774996/how-to-compare-show-the-difference-between-2-videos-in-ffmpeg">this answer</a>, which shows how to blend two video streams together and display the difference.  Doing so "works" in the sense that I get a solid green screen with my duplicate files (indicating no pixel differences), but I can't use that in a script to handle hundreds of comparisons because I still have to visually inspect the result.</p>

<p>Is there a way to get ffmpeg to give a yes/no answer?</p>

<p>I've seen <a href="https://video.stackexchange.com/questions/5307/how-can-i-find-duplicate-videos-from-a-large-set-of-videos-of-varying-formats">this answer</a>, but found the answer to be entirely too manual.  I need an automated process that can go through large filesets in one run.   </p>

<video><ffmpeg>
Tool to find Duplicate Video files ignoring tags
rothloup
# Tool to find Duplicate Video files ignoring tags

(Asked by: username_0 on 2018-10-28)

I am looking for an automated method to identify duplicated video files from my large collection of videos.

I have used fslint to do this in the past, but I started fixing the EXIF tags on my videos, which makes fslint identify them as different files. The only difference between the original file and the modified files are the EXIF tags, which in most cases have simply been time-shifted to correct for timezone or camera offset. The videos have not had any encoding or conversion of any type.

Is there a tool something that I can use to identify duplicate videos based only on the AV streams, i.e. ignoring the tags? I'm essentially looking to duplicate fslint-like functionality, but ignoring the EXIF tags. Does such a tool exist?

I have a suspicion that ffmpeg can be used to do this, but all I've been able to find so far is [this answer](https://stackoverflow.com/questions/25774996/how-to-compare-show-the-difference-between-2-videos-in-ffmpeg), which shows how to blend two video streams together and display the difference. Doing so "works" in the sense that I get a solid green screen with my duplicate files (indicating no pixel differences), but I can't use that in a script to handle hundreds of comparisons because I still have to visually inspect the result.

Is there a way to get ffmpeg to give a yes/no answer?

I've seen [this answer](https://video.stackexchange.com/questions/5307/how-can-i-find-duplicate-videos-from-a-large-set-of-videos-of-varying-formats), but found the answer to be entirely too manual. I need an automated process that can go through large filesets in one run.

**Tags:** video, ffmpeg


---

## 2 answers

### Answer by username_1 on 2021-03-25. Score: 1

It might be overkill in your situation since the videos aren't actually different resolutions, frame rates and or codecs, but still :

As with another answer <https://video.stackexchange.com/a/33521/32436> :

> You could check out <https://theophanemayaud.github.io/video-simili-duplicate-cleaner/> (macOS and Windows binaries available) It's open source, based on vidupe and QT and relies on ffmpeg.
>
> The idea is to extract a few (eg 2, the first and the last) frames, and compare them. If the duration is close, and the frames look similar : you've got a duplicate.
>
> It uses opencv for frame comparison, and ffmpeg for frame extraction (ffmpeg is also for video metadata which is very useful when trying to tell which video to keep !). Since it's open source, you can improve it to fit your needs !

Because it looks at the video data, you shouldn't have problems of tags or other metadata that has changed !


---

### Answer by username_2 on 2018-11-25. Score: 0

I had the same question and I managed to answer it by creating an open source program around ffmpeg and findimagedupes. Feel free to try it :
<https://github.com/username_2Crette/VideoDedup/blob/master/README.md>


---

