<p>As I'm working on the exact same challenge, I'm currently thinking of abusing the <a href="https://ffmpeg.org/ffmpeg-filters.html#vidstabtransform-1" rel="nofollow">ffmpeg vidstabtransform</a> filter with a generated trf file. But I have not worked out the details yet.</p>

<p><a href="https://github.com/georgmartius/vid.stab/blob/master/src/serialize.c" rel="nofollow">Info on the trf file format.</a></p>

CC BY-SA 3.0
2015-05-02T01:57:46.293Z
2015-05-02T01:57:46.293Z
Jannes
<p>Similar to Gyan's overlay answer, a slightly simpler version can use a basic -filter_script with the "swaprect" and "crop" filters.  Move the crop target to 0,0 and then crop the target area at 0,0.  The target has to be swapped to 0,0 as otherwise the other swapped rect can overlap.  This worked with ffmpeg version 3.4.6-0ubuntu0.18.04.1</p>

<p>Make sure the frame numbers are relative to the offset passed with "-ss".</p>

<p>An example targeting 1280x720 crops from a 4096x2160 source.  The source is 24fps and the tracking info is 8fps.</p>

<p>crop-filter-script:</p>

<pre><code>swaprect=1280:720:0:0:1568:594:enable='between(n,0,26)',
swaprect=1280:720:0:0:1552:598:enable='between(n,27,29)',
swaprect=1280:720:0:0:1565:583:enable='between(n,30,32)',
swaprect=1280:720:0:0:1603:576:enable='between(n,33,35)',
crop=1280:720:0:0
</code></pre>

<p><code>ffmpeg -ss [start offset] -t [duration] -i input.mov -filter_script:v:0 crop-filter-script -acodec copy out.mov</code></p>

CC BY-SA 4.0
2019-12-30T22:41:29.240Z
2019-12-31T18:25:25.830Z
2019-12-31T18:25:25.830Z
jason gilbert
<p>I have already changed to <code>avconv</code>, so i apologise if the answer may be a little different in the classic ffmpeg, but i believe that not much difference should be there.</p>

<p>You can have a moving crop if you can create some sort of formula between the frame number and the position of the crop; but there are no analytic tools available to do that. This means that if the the speaker moves in a predictable way you could theoretically do that without an external application.</p>

<p>Let's assume, for example, that the speaker is moving from top-left to bottom-right in the picture. You could do something along the lines of:</p>

<pre><code>avconv -i input.ogv -vf 'crop=200:100:n:n' -c:v libtheora output.ogv
</code></pre>

<p>In here i'm moving 1 pixel per frame, which is very unlikely how somebody would move.</p>

<p>Keep in mind that there are quite a lot of math functions that are provided by the filter interface of avconv (and i'm sure ffmpeg is similar).</p>

<p>So let's imagine that you want to move 1 pixel per frame, but only from 200 to 350 frame. You can then do</p>

<pre><code>min(max(n,200)-200, 150)
</code></pre>

<p>For the frames &lt;200, <code>max(n,200)</code> would generate 200, then we take away 200 to make it 0, and then <code>min</code> will return the first part up until 350 frame, where that result would become more than 150 and the constant will begin to be returned.</p>

<p>Although writing such formula by hand would be quite tedious, maybe it will be helpful in some cases as well.</p>

<p>Having said all that, if i were to have an option of having access to the software that is able to work on images frame by frame, i would go that route.</p>

CC BY-SA 3.0
2015-04-27T10:43:41.860Z
2015-04-27T16:22:29.227Z
2015-04-27T16:22:29.227Z
v010dya
<p>Given that you have "<em>a file that contains the cropping rectangle's X, Y, Width, &amp; Height for each frame</em>", you can use the filter_script option to do this in FFmpeg. The crop filter does not support timeline editing but the overlay filter does. So, if you create a blank canvas of the same resolution as your video and then overlay the video on top with co-ordinates such that your intended region is superimposed in the visible area, the goal is achieved.</p>

<p>Basic syntax is</p>

<pre><code>ffmpeg -i in.mp4 -filter_complex_script file.txt -map "[out]" output.mp4
</code></pre>

<p>where <code>file.txt</code> looks like this:</p>

<pre><code>nullsrc=WxH:r=FPS[cv];
[cv][0]overlay=-X0:-Y0:shortest=1:enable='eq(n\,0)'[b0];
[b0][0]overlay=-X1:-Y1:shortest=1:enable='eq(n\,1)'[b1];
[b1][0]overlay=-X2:-Y2:shortest=1:enable='eq(n\,2)'[b2];
...
[bm-1][0]overlay=-Xm:-Ym:shortest=1:enable='eq(n\,m)'[out]
</code></pre>

<p>The canvas <code>W</code> and <code>H</code> should be equal to the (fixed/static) crop dimensions and of the same <code>FPS</code>. Then each frame of the video is overlaid at (<strong>-</strong> X, <strong>-</strong> Y) so that the top-left corner of the desired region is at (0,0) of the canvas. Each overlay occurs for one frame. For efficiency's sake, if you have intervals where the crop region is static, you should use the between evaluator e.g.</p>

<pre><code>[bi][0]overlay=-Xi:-Yi:shortest=1:enable='between(n\,1200\,1445)'[bj];
</code></pre>

<p>You can also evaluate using timestamp <code>t</code> instead of frame index.</p>

CC BY-SA 3.0
2016-10-03T10:11:48.500Z
I tried implementing this solution to see how it goes, just wanted a brute version at the beginning for testing so I haven't implemented between, I just added the position for each frame, for the first 3000 frames. My ffmpeg memory is now at 36GB and going with 1 frame per minute :( I guess this isn't the best solution for per frame haha
Eek
CC BY-SA 3.0
2016-10-03T15:17:24.843Z
I'm currently out of town,  but look into using trim filter to isolate one frame and output to Png. So n commands for n frames. Then concat the images to a video
Gyan
CC BY-SA 3.0
2016-09-16T13:04:05.073Z
2016-09-16T13:17:46.907Z
2016-09-16T13:17:46.907Z
Gyan
<p>I have an HD video of a speaker giving a lecture.  I would like to resize the video by cropping each frame, but the cropping rectangle needs to be centered on the speaker's head as he moves around the stage.</p>

<p>I <strong>can</strong> generate a file that contains the cropping rectangle's X, Y, Width, &amp; Height for each frame in the video.</p>

<p>What's the best way to apply this moving crop to the video?</p>

<p>It should work if I..</p>

<ol>
<li>Use ffmpeg to extract each frame to a folder</li>
<li>Use another program to crop each image file</li>
<li>Use ffmpeg to rebuild the video using the cropped image files</li>
</ol>

<p>Is there a better way to do this?</p>

CC BY-SA 3.0
2015-04-29T02:43:59.700Z
ImageMagick could do the cropping if you have numerical description of the crop frame. I'd suggest: export as image sequence using ffmpeg, batch process using ImageMagick and remux as a movie in ffmpeg. You'll need some sort of shell scripting to glue it all together probably. If you post an example of the file with the crop info I can work out the details for you.
stib
CC BY-SA 3.0
2015-05-02T01:51:07.777Z
May I ask how you generate that file? I'm working on exactly the same issue. I have a proof of concept using the ffmpeg tblend subtract filter and then the cropdetect filter. Wondering if you have a better way.
Jannes
<ffmpeg>
Moving Crop in Video
David
# Moving Crop in Video

(Asked by: username_0 on 2015-04-24)

I have an HD video of a speaker giving a lecture. I would like to resize the video by cropping each frame, but the cropping rectangle needs to be centered on the speaker's head as he moves around the stage.

I **can** generate a file that contains the cropping rectangle's X, Y, Width, & Height for each frame in the video.

What's the best way to apply this moving crop to the video?

It should work if I..

1. Use ffmpeg to extract each frame to a folder
2. Use another program to crop each image file
3. Use ffmpeg to rebuild the video using the cropped image files

Is there a better way to do this?

**Tags:** ffmpeg

> **Comments:**
> **username_1**: ImageMagick could do the cropping if you have numerical description of the crop frame. I'd suggest: export as image sequence using ffmpeg, batch process using ImageMagick and remux as a movie in ffmpeg. You'll need some sort of shell scripting to glue it all together probably. If you post an example of the file with the crop info I can work out the details for you.  
> **username_2**: May I ask how you generate that file? I'm working on exactly the same issue. I have a proof of concept using the ffmpeg tblend subtract filter and then the cropdetect filter. Wondering if you have a better way.  

---

## 4 answers

### Answer by username_3 on 2019-12-30. Score: 3

Similar to username_5's overlay answer, a slightly simpler version can use a basic -filter_script with the "swaprect" and "crop" filters.  Move the crop target to 0,0 and then crop the target area at 0,0.  The target has to be swapped to 0,0 as otherwise the other swapped rect can overlap.  This worked with ffmpeg version 3.4.6-0ubuntu0.18.04.1
Make sure the frame numbers are relative to the offset passed with "-ss".
An example targeting 1280x720 crops from a 4096x2160 source.  The source is 24fps and the tracking info is 8fps.
crop-filter-script:
swaprect=1280:720:0:0:1568:594:enable='between(n,0,26)',
swaprect=1280:720:0:0:1552:598:enable='between(n,27,29)',
swaprect=1280:720:0:0:1565:583:enable='between(n,30,32)',
swaprect=1280:720:0:0:1603:576:enable='between(n,33,35)',
crop=1280:720:0:0

ffmpeg -ss [start offset] -t [duration] -i input.mov -filter_script:v:0 crop-filter-script -acodec copy out.mov


---

### Answer by username_2 on 2015-05-02. Score: 2

As I'm working on the exact same challenge, I'm currently thinking of abusing the [ffmpeg vidstabtransform](https://ffmpeg.org/ffmpeg-filters.html#vidstabtransform-1) filter with a generated trf file. But I have not worked out the details yet.

[Info on the trf file format.](https://github.com/georgmartius/vid.stab/blob/master/src/serialize.c)


---

### Answer by username_4 on 2015-04-27. Score: 1

I have already changed to avconv, so i apologise if the answer may be a little different in the classic ffmpeg, but i believe that not much difference should be there.
You can have a moving crop if you can create some sort of formula between the frame number and the position of the crop; but there are no analytic tools available to do that. This means that if the the speaker moves in a predictable way you could theoretically do that without an external application.
Let's assume, for example, that the speaker is moving from top-left to bottom-right in the picture. You could do something along the lines of:
avconv -i input.ogv -vf 'crop=200:100:n:n' -c:v libtheora output.ogv

In here i'm moving 1 pixel per frame, which is very unlikely how somebody would move.
Keep in mind that there are quite a lot of math functions that are provided by the filter interface of avconv (and i'm sure ffmpeg is similar).
So let's imagine that you want to move 1 pixel per frame, but only from 200 to 350 frame. You can then do
min(max(n,200)-200, 150)

For the frames <200, max(n,200) would generate 200, then we take away 200 to make it 0, and then min will return the first part up until 350 frame, where that result would become more than 150 and the constant will begin to be returned.
Although writing such formula by hand would be quite tedious, maybe it will be helpful in some cases as well.
Having said all that, if i were to have an option of having access to the software that is able to work on images frame by frame, i would go that route.


---

### Answer by username_5 on 2016-09-16. Score: 1

Given that you have "a file that contains the cropping rectangle's X, Y, Width, & Height for each frame", you can use the filter_script option to do this in FFmpeg. The crop filter does not support timeline editing but the overlay filter does. So, if you create a blank canvas of the same resolution as your video and then overlay the video on top with co-ordinates such that your intended region is superimposed in the visible area, the goal is achieved.
Basic syntax is
ffmpeg -i in.mp4 -filter_complex_script file.txt -map "[out]" output.mp4

where file.txt looks like this:
nullsrc=WxH:r=FPS[cv];
[cv][0]overlay=-X0:-Y0:shortest=1:enable='eq(n\,0)'[b0];
[b0][0]overlay=-X1:-Y1:shortest=1:enable='eq(n\,1)'[b1];
[b1][0]overlay=-X2:-Y2:shortest=1:enable='eq(n\,2)'[b2];
...
[bm-1][0]overlay=-Xm:-Ym:shortest=1:enable='eq(n\,m)'[out]

The canvas W and H should be equal to the (fixed/static) crop dimensions and of the same FPS. Then each frame of the video is overlaid at (- X, - Y) so that the top-left corner of the desired region is at (0,0) of the canvas. Each overlay occurs for one frame. For efficiency's sake, if you have intervals where the crop region is static, you should use the between evaluator e.g.
[bi][0]overlay=-Xi:-Yi:shortest=1:enable='between(n\,1200\,1445)'[bj];

You can also evaluate using timestamp t instead of frame index.

> **Comments:**
> **username_6**: I tried implementing this solution to see how it goes, just wanted a brute version at the beginning for testing so I haven't implemented between, I just added the position for each frame, for the first 3000 frames. My ffmpeg memory is now at 36GB and going with 1 frame per minute :( I guess this isn't the best solution for per frame haha  
> **username_5**: I'm currently out of town,  but look into using trim filter to isolate one frame and output to Png. So n commands for n frames. Then concat the images to a video  

---

