<p>Can't comment on the soundness of your theory, but here's a way to do it. Not directly, but in a roundabout way.</p>

<p>FFmpeg's segment muxer can break up a file at GOP boundaries, so running the command below generates a set of videos which each start with a keyframe but contain no other keyframes.</p>

<pre><code>ffmpeg -i test.mp4 -c copy -f segment -segment_time 0.001 -segment_list list.ffcat -reset_timestamps 1 segs%d.mp4
</code></pre>

<p>This will create</p>

<pre><code>segs0.mp4
segs1.mp4
...
segs101.mp4
..etc
</code></pre>

<p>Extract each segment to images</p>

<pre><code>ffmpeg -i segN.mp4 outN-%d.png
</code></pre>

<p>Apply A on first frame of each image series and B on the rest.</p>

<p>Then for each processed series, run</p>

<pre><code>ffmpeg -framerate X -i procN-%d.png -c:v libx264 -keyint_min 1000 -x264opts stitchable "result\segsN.mp4"
</code></pre>

<p><code>-keyint_min</code> value should be greater than no. of frames in each series.</p>

<p>Hopefully the names of the resulting MP4s are the same as the original split segments, which makes the next step more convenient.</p>

<p>Copy the <code>list.ffcat</code> created in the initial segmenting command to the results folder and run</p>

<pre><code>ffmpeg -f concat -safe 0 -i list.ffcat -c copy processed.mp4
</code></pre>

CC BY-SA 3.0
2017-04-28T19:09:50.020Z
This works but my theory needs more work ;) I can clearly see "the switch" between filter modes so next step is to see if I could maybe apply it to every 10th frame or something like that. Thanks!
jimmy
CC BY-SA 4.0
2017-04-28T17:19:58.280Z
2020-03-11T13:45:04.200Z
2020-03-11T13:45:04.200Z
Gyan
<p>I have a research project where I need to apply very, very computationally heavy image filters on a h264 video. Due some restrictions outside my control I have to use proprietary filter that works only on images and not video. Filter has two modes, A mode is very computationally heavy but produces production ready quality and B mode is fast but produces less than stellar quality.</p>

<p>My theory is that given how h264 uses I-frames to get the reference and rest of the frames are "non-essential", if I can find apply the high quality A mode on I-frames and B mode on other frames, I should be able to leverage h264's own compression to get the result without having to use filter on A mode to every frame. Another avenue would be to try with h265 because it's more robust frame prediction capabilities.</p>

<p>What I've figured out so far is that I need to find a way to extract all frames from a video while knowing which ones are I-frames, then do a filter on all the frames, and then re-encode the video back together with minimum quality loss.</p>

<p><strong>So first question is that how can I extract frames with ffmpeg so that it somehow identifies each frame type on the filename?</strong></p>

<pre><code>ffmpeg -i C:\test.mp4 -vf select='eq(pict_type\,I)',setpts='N/(25*TB)' C:\testTemp\%09d.jpg
</code></pre>

<p>Above gives you i-frames but I haven't figured out how I can get this to export all the frames and to put the type on the filename?</p>

<p><strong>Second question is that how can I re-encode the video so that it keeps the I-frame positions?</strong></p>

<pre><code>ffmpeg -framerate 24 -i frame-%03d.png output.mp4
</code></pre>

<p>Above would make the video out of frames but the I-frame positioning eludes me.</p>

<p>I know this is a complex problem so all help is truly appreciated!</p>

<video><ffmpeg><h.264><h265>
How to extract frames and saving frame type to filenames with ffmpeg?
jimmy
# How to extract frames and saving frame type to filenames with ffmpeg?

(Asked by: username_0 on 2017-04-28)

I have a research project where I need to apply very, very computationally heavy image filters on a h264 video. Due some restrictions outside my control I have to use proprietary filter that works only on images and not video. Filter has two modes, A mode is very computationally heavy but produces production ready quality and B mode is fast but produces less than stellar quality.
My theory is that given how h264 uses I-frames to get the reference and rest of the frames are "non-essential", if I can find apply the high quality A mode on I-frames and B mode on other frames, I should be able to leverage h264's own compression to get the result without having to use filter on A mode to every frame. Another avenue would be to try with h265 because it's more robust frame prediction capabilities.
What I've figured out so far is that I need to find a way to extract all frames from a video while knowing which ones are I-frames, then do a filter on all the frames, and then re-encode the video back together with minimum quality loss.
So first question is that how can I extract frames with ffmpeg so that it somehow identifies each frame type on the filename?
ffmpeg -i C:\test.mp4 -vf select='eq(pict_type\,I)',setpts='N/(25*TB)' C:\testTemp\%09d.jpg

Above gives you i-frames but I haven't figured out how I can get this to export all the frames and to put the type on the filename?
Second question is that how can I re-encode the video so that it keeps the I-frame positions?
ffmpeg -framerate 24 -i frame-%03d.png output.mp4

Above would make the video out of frames but the I-frame positioning eludes me.
I know this is a complex problem so all help is truly appreciated!

**Tags:** video, ffmpeg, h.264, h265


---

## 1 answers

### Answer by username_1 on 2017-04-28. Score: 2 | Accepted answer

Can't comment on the soundness of your theory, but here's a way to do it. Not directly, but in a roundabout way.
FFmpeg's segment muxer can break up a file at GOP boundaries, so running the command below generates a set of videos which each start with a keyframe but contain no other keyframes.
ffmpeg -i test.mp4 -c copy -f segment -segment_time 0.001 -segment_list list.ffcat -reset_timestamps 1 segs%d.mp4

This will create
segs0.mp4
segs1.mp4
...
segs101.mp4
..etc

Extract each segment to images
ffmpeg -i segN.mp4 outN-%d.png

Apply A on first frame of each image series and B on the rest.
Then for each processed series, run
ffmpeg -framerate X -i procN-%d.png -c:v libx264 -keyint_min 1000 -x264opts stitchable "result\segsN.mp4"

-keyint_min value should be greater than no. of frames in each series.
Hopefully the names of the resulting MP4s are the same as the original split segments, which makes the next step more convenient.
Copy the list.ffcat created in the initial segmenting command to the results folder and run
ffmpeg -f concat -safe 0 -i list.ffcat -c copy processed.mp4

> **Comments:**
> **username_0**: This works but my theory needs more work ;) I can clearly see "the switch" between filter modes so next step is to see if I could maybe apply it to every 10th frame or something like that. Thanks!  

---

