<p>I wrote the following script to solve my problem. It uses <code>framemd5</code> checksums to find the inpoints in time and then uses the <code>concat</code> demuxer (thanks @Gyan for the tip!).</p>
<pre class="lang-py prettyprint-override"><code># Usage: ffmpeg $(python3 script.py input1.mp4 input2.mp4) -y -an output.mp4

import os
import sys
import subprocess

ffmpeg_concat_echo, md5_last_frame = '', None

for video_path in map(os.path.abspath, sys.argv[1:]):
        framemd5 = subprocess.check_output(['ffmpeg', '-nostats', '-hide_banner', '-i', video_path, '-an', '-f', 'framemd5', '-c', 'copy', '-'], stderr = open(os.devnull, 'w'))
        tb_num, tb_den = 1, 1
        inpoint = 0.0
        for line in framemd5.split(b'\n'):
                if line.startswith(b'#tb'):
                        tb_num, tb_den = list(map(int, line.split()[-1].split(b'/')))
                elif line and not line.startswith(b'#'):
                        splitted = line.split(b',')
                        md5 = splitted[-1].strip()
                        pts_time = float(splitted[2]) * tb_num / tb_den
                        if md5_last_frame is not None and md5 == md5_last_frame:
                                inpoint = pts_time
        md5_last_frame = md5
        ffmpeg_concat_echo += 'echo &quot;file {}&quot;; echo &quot;inpoint {}&quot;;'.format(video_path, inpoint);

print('-nostats -hide_banner -avoid_negative_ts make_zero -fflags +genpts -f concat -safe 0 -i &lt;({}) -c copy'.format(ffmpeg_concat_echo))
</code></pre>
<p>Below is another version (<a href="https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18" rel="nofollow noreferrer">source</a>, <a href="https://web.archive.org/web/20230311234256/https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18" rel="nofollow noreferrer">archived</a>) which supports exact PTS inpoints using <code>concat</code> video filter; computing inpoints using explicit frame differences.</p>
<pre class="lang-py prettyprint-override"><code># Usage: ffmpeg $(( echo input1.mp4; echo input2.mp4 ) | python3 concat_overlapping_video_framemd5_demuxer.py concat_file.txt) -y -an output.mp4

import os
import sys
import subprocess

ffmpeg_concat_file, md5_last_frame = open(sys.argv[1], 'w'), None
for video_path in map(os.path.abspath, list(map(str.strip, sys.stdin))):
        framemd5 = subprocess.check_output(['ffmpeg', '-nostats', '-hide_banner', '-i', video_path, '-an', '-f', 'framemd5', '-c', 'copy', '-'], stderr = open(os.devnull, 'w'))
        inpoint, tb_num, tb_den = 0, 1, 1
        for line in framemd5.split(b'\n'):
                if line.startswith(b'#tb'):
                        tb_num, tb_den = list(map(int, line.split()[-1].split(b'/')))
                elif not line.startswith(b'#') and len(line) &gt; 0:
                        splitted = line.split(b',')
                        md5, pts_time = splitted[-1].strip(), float(splitted[2]) * tb_num / tb_den
                        if md5 == md5_last_frame:
                                inpoint = pts_time
        md5_last_frame = md5
        ffmpeg_concat_file.write('file {}\ninpoint {}\n'.format(video_path, inpoint))

print('-nostats -hide_banner -avoid_negative_ts make_zero -fflags +genpts -f concat -safe 0 -i {} -c copy'.format(sys.argv[1]))
</code></pre>

CC BY-SA 4.0
2018-08-22T09:19:42.240Z
This may result in "Non-monotonous DTS" errors. Some ways to solve:
1) Use `concat` video filter. Downsides: will re-encode
2) [Extract streams and re-assemble them back](https://stackoverflow.com/questions/31691943/ffmpeg-concat-poduces-dts-out-of-order-errors). Downsides: will make a file copy, unless [named pipes are used](https://trac.ffmpeg.org/wiki/Concatenate#Usingnamedpipestoavoidintermediatefiles)
3) Wait until [this patch](http://ffmpeg.org/pipermail/ffmpeg-devel/2018-March/226706.html) gets merged. Hopefully it would fix the problem
Vadim Kantorov
CC BY-SA 4.0
2019-06-05T05:10:29.550Z
is there better version of it, gist isn't working
Muhammad Umer
CC BY-SA 4.0
2020-10-21T15:40:22.103Z
This worked great for stitching some security camera footage together, thank you.
hannasm
CC BY-SA 4.0
2021-01-06T23:02:55.547Z
The gist link is broken, could you please share the code again in a more permanent way?
Isti115
CC BY-SA 4.0
2023-03-11T23:48:35.217Z
[Revised gist](https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18) ([archived](https://web.archive.org/web/20230311234256/https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18)), [original gist](https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18/321290839cde9a33efa22ad458e5e653de1e5a97) ([archived](https://web.archive.org/web/20230311234353/https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18/321290839cde9a33efa22ad458e5e653de1e5a97)), [revisions](https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18/revisions).
mrienstra
CC BY-SA 4.0
2023-03-11T23:49:59.143Z
Continued from [above](https://video.stackexchange.com/questions/24681/concat-overlapping-videos-with-ffmpeg-using-python3#comment46618_24689): [archived revisions](https://web.archive.org/web/20230311234407/https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18/revisions).
mrienstra
CC BY-SA 4.0
2018-08-21T14:03:53.893Z
2023-03-18T19:30:56.337Z
2023-03-18T19:30:56.337Z
Vadim Kantorov
<p>I have a number of videos that overlap in time that were downloaded from an online video stream:
<code>
0m00s  - 15m10s
14m50s - 30m20s
...
</code>
The first few seconds (usually around 20 seconds) of a video segment duplicate last few seconds of the previous segment.</p>

<p>I'd like to concat the video segments and form a single video file without duplicate parts. I can do it programmatically using <code>ffmpeg</code> libraries to find the cut points by minimizing error between a frame of the current segment and the last frame of the previous segment.</p>

<p>Is there a way to achieve this with an ffmpeg command?</p>

CC BY-SA 4.0
2018-08-20T10:37:50.073Z
If the files have absolute timestamps, it may be possible. But if no, it's not possible in a single command. You can run one command to find join point betweeen two consecutive segments using blend filter/blackframe and then write a concat demuxer text file with the required inpoint/outpoint
Gyan
CC BY-SA 4.0
2018-08-20T10:43:26.143Z
I just checked with `-vf showinfo`, and it seems they do not have absolute timestamps, but they do have some checksums (are these useful for finding exact duplicates?)

Thanks for the pointer on blend/blackframe - googling it.
Vadim Kantorov
CC BY-SA 4.0
2018-08-20T10:46:35.337Z
You need to add `-copyts` else ffmpeg will remove any starting offset before it gets to the filter.
Gyan
CC BY-SA 4.0
2018-08-20T12:56:31.457Z
It seems `framemd5` matching is enough for my videos. I can make `framemd5` files for every video segment and fetch out dts / pts of the starting frame in every segment. Is there a way to concat with dts / pts inpoints instead of timestamps?
Vadim Kantorov
CC BY-SA 4.0
2018-08-20T14:23:49.690Z
No, time has to be in seconds.
Gyan
CC BY-SA 4.0
2018-08-20T22:10:24.617Z
I found that `trim` filter has `start_pts` option, but I can't make it work with `pts` values extracted from `framemd5` output (`start` with seconds works, but `start_pts` does not)
Vadim Kantorov
CC BY-SA 4.0
2023-03-17T23:33:49.213Z
If someone lands here needing a "fuzzier" match than `framemd5`, see https://superuser.com/questions/663731/find-video-file-position-using-an-image-match
mrienstra
<ffmpeg>
Concat overlapping videos with ffmpeg using python3
Vadim Kantorov
# Concat overlapping videos with ffmpeg using python3

(Asked by: username_0 on 2018-08-20)

I have a number of videos that overlap in time that were downloaded from an online video stream:
`0m00s - 15m10s
14m50s - 30m20s
...`
The first few seconds (usually around 20 seconds) of a video segment duplicate last few seconds of the previous segment.

I'd like to concat the video segments and form a single video file without duplicate parts. I can do it programmatically using `ffmpeg` libraries to find the cut points by minimizing error between a frame of the current segment and the last frame of the previous segment.

Is there a way to achieve this with an ffmpeg command?

**Tags:** ffmpeg

> **Comments:**
> **username_1**: If the files have absolute timestamps, it may be possible. But if no, it's not possible in a single command. You can run one command to find join point betweeen two consecutive segments using blend filter/blackframe and then write a concat demuxer text file with the required inpoint/outpoint  
> **username_0**: I just checked with `-vf showinfo`, and it seems they do not have absolute timestamps, but they do have some checksums (are these useful for finding exact duplicates?)  Thanks for the pointer on blend/blackframe - googling it.  
> **username_1**: You need to add `-copyts` else ffmpeg will remove any starting offset before it gets to the filter.  
> **username_0**: It seems `framemd5` matching is enough for my videos. I can make `framemd5` files for every video segment and fetch out dts / pts of the starting frame in every segment. Is there a way to concat with dts / pts inpoints instead of timestamps?  
> **username_1**: No, time has to be in seconds.  
> **username_0**: I found that `trim` filter has `start_pts` option, but I can't make it work with `pts` values extracted from `framemd5` output (`start` with seconds works, but `start_pts` does not)  
> **username_2**: If someone lands here needing a "fuzzier" match than `framemd5`, see https://superuser.com/questions/663731/find-video-file-position-using-an-image-match  

---

## 1 answers

### Answer by username_0 on 2018-08-21. Score: 1 | Accepted answer

I wrote the following script to solve my problem. It uses framemd5 checksums to find the inpoints in time and then uses the concat demuxer (thanks @username_1 for the tip!).
# Usage: ffmpeg $(python3 script.py input1.mp4 input2.mp4) -y -an output.mp4

import os
import sys
import subprocess

ffmpeg_concat_echo, md5_last_frame = '', None

for video_path in map(os.path.abspath, sys.argv[1:]):
        framemd5 = subprocess.check_output(['ffmpeg', '-nostats', '-hide_banner', '-i', video_path, '-an', '-f', 'framemd5', '-c', 'copy', '-'], stderr = open(os.devnull, 'w'))
        tb_num, tb_den = 1, 1
        inpoint = 0.0
        for line in framemd5.split(b'\n'):
                if line.startswith(b'#tb'):
                        tb_num, tb_den = list(map(int, line.split()[-1].split(b'/')))
                elif line and not line.startswith(b'#'):
                        splitted = line.split(b',')
                        md5 = splitted[-1].strip()
                        pts_time = float(splitted[2]) * tb_num / tb_den
                        if md5_last_frame is not None and md5 == md5_last_frame:
                                inpoint = pts_time
        md5_last_frame = md5
        ffmpeg_concat_echo += 'echo "file {}"; echo "inpoint {}";'.format(video_path, inpoint);

print('-nostats -hide_banner -avoid_negative_ts make_zero -fflags +genpts -f concat -safe 0 -i <({}) -c copy'.format(ffmpeg_concat_echo))

Below is another version (source, archived) which supports exact PTS inpoints using concat video filter; computing inpoints using explicit frame differences.
# Usage: ffmpeg $(( echo input1.mp4; echo input2.mp4 ) | python3 concat_overlapping_video_framemd5_demuxer.py concat_file.txt) -y -an output.mp4

import os
import sys
import subprocess

ffmpeg_concat_file, md5_last_frame = open(sys.argv[1], 'w'), None
for video_path in map(os.path.abspath, list(map(str.strip, sys.stdin))):
        framemd5 = subprocess.check_output(['ffmpeg', '-nostats', '-hide_banner', '-i', video_path, '-an', '-f', 'framemd5', '-c', 'copy', '-'], stderr = open(os.devnull, 'w'))
        inpoint, tb_num, tb_den = 0, 1, 1
        for line in framemd5.split(b'\n'):
                if line.startswith(b'#tb'):
                        tb_num, tb_den = list(map(int, line.split()[-1].split(b'/')))
                elif not line.startswith(b'#') and len(line) > 0:
                        splitted = line.split(b',')
                        md5, pts_time = splitted[-1].strip(), float(splitted[2]) * tb_num / tb_den
                        if md5 == md5_last_frame:
                                inpoint = pts_time
        md5_last_frame = md5
        ffmpeg_concat_file.write('file {}\ninpoint {}\n'.format(video_path, inpoint))

print('-nostats -hide_banner -avoid_negative_ts make_zero -fflags +genpts -f concat -safe 0 -i {} -c copy'.format(sys.argv[1]))

> **Comments:**
> **username_0**: This may result in "Non-monotonous DTS" errors. Some ways to solve: 1) Use `concat` video filter. Downsides: will re-encode 2) [Extract streams and re-assemble them back](https://stackoverflow.com/questions/31691943/ffmpeg-concat-poduces-dts-out-of-order-errors). Downsides: will make a file copy, unless [named pipes are used](https://trac.ffmpeg.org/wiki/Concatenate#Usingnamedpipestoavoidintermediatefiles) 3) Wait until [this patch](http://ffmpeg.org/pipermail/ffmpeg-devel/2018-March/226706.html) gets merged. Hopefully it would fix the problem  
> **username_3**: is there better version of it, gist isn't working  
> **username_4**: This worked great for stitching some security camera footage together, thank you.  
> **username_5**: The gist link is broken, could you please share the code again in a more permanent way?  
> **username_2**: [Revised gist](https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18) ([archived](https://web.archive.org/web/20230311234256/https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18)), [original gist](https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18/321290839cde9a33efa22ad458e5e653de1e5a97) ([archived](https://web.archive.org/web/20230311234353/https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18/321290839cde9a33efa22ad458e5e653de1e5a97)), [revisions](https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18/revisions).  
> **username_2**: Continued from [above](https://video.stackexchange.com/questions/24681/concat-overlapping-videos-with-ffmpeg-using-python3#comment46618_24689): [archived revisions](https://web.archive.org/web/20230311234407/https://gist.github.com/yar/5056e5bd303a23de98139365d2ff5d18/revisions).  

---

