<p>You could do this in After Effects.  There are lots of examples online where people use Expressions to examine the average brightness of each frame, then act upon it.  For example, you could add a marker to each frame that fell below a certain threshold level. Or make the frame's opacity go to 0%, to allow another track to show through from behind.</p>

<p>Tutorial examples here:
<a href="http://markos.co.nz/2012/05/after-effects-expression-use-the-luminance-of-one-layer-to-drive-another-layer/" rel="nofollow noreferrer">http://markos.co.nz/2012/05/after-effects-expression-use-the-luminance-of-one-layer-to-drive-another-layer/</a></p>

<p><a href="http://blogs.adobe.com/aftereffects/2009/07/color-sampler-using-sampleimag.html" rel="nofollow noreferrer">http://blogs.adobe.com/aftereffects/2009/07/color-sampler-using-sampleimag.html</a></p>

<p>Here's an example expression I wrote based on the blog posts above -</p>

<pre><code>threshold = 0.01;  // this is the threshold below which the track becomes transparent - black = 0.
driverLayer = thisComp.layer("Pre-comp 1");
samplePoint = [0,0]; // examine from the top left pixel
sampleSize = [640,360]; // examine the full comp size (in this case 640x360)
lightnessSample = driverLayer.sampleImage(samplePoint,sampleSize);  // this samples the image as rgb
driverLightness = rgbToHsl(lightnessSample)[2]; // converts to hue, saturation and lightness, then only takes lightness
if (driverLightness &gt;=threshold) transform.opacity = 100 else 0; // tell opacity to be either 100 or zero depending upon the threshold point chosen in line 1
</code></pre>

<p>I applied this to the opacity layer.  You would need to adjust the threshold value, and the size of your comp ( in my case 640x360). (the red solid below was to see whether the opacity change was working.  I guess you would put a copy of your video showing the previous frame here?  Will look as though the video got stuck though...</p>

<p><img src="https://i.sstatic.net/wLUD5.png" alt="enter image description here"></p>

CC BY-SA 3.0
2015-01-01T13:22:24.293Z
2015-01-01T14:17:58.993Z
2015-01-01T14:17:58.993Z
tomh
<p>My video file has some blackish frames (it's almost black, but not exactly black, i.e. #000000), and I want to filter out them, replacing them with neighboring frames.</p>

<p>However, I can't find a simple way to do that.</p>

<p>Of course, I can extract all the frame images from the video file with FFmpeg:</p>

<p><code>ffmpeg -i foo.mp4 -an -framerate 30 -s 1280x720 -f image2 foo-%05d.jpeg</code>  </p>

<p>and then list all the blackish images with some image tools, replace them with neighboring-numbered files, and merge them:</p>

<p><code>ffmpeg -i foo.mp4 -vn -acodec copy foo.m4a</code><br>
<code>ffmpeg -i foo.m4a -f image2 -framerate 30 -i foo-%05d.jpeg -r 30 -s 1280x720 output.mp4</code></p>

<p>but this will create over 10,000 intermediate image files and it is also time-consuming.</p>

<p>Is there any software or programming language that helps this work?</p>

<p>I am using Windows x64.</p>

CC BY-SA 3.0
2015-01-01T12:13:47.287Z
Remember that to do the image processing, whatever software you use is going to have to decode each frame so it's always going to be time consuming. Writing them to disk as separate image files will take more time I suppose, but you could perhaps use something like a named pipe to keep it in memory, depending on what image tools you use.
stib
CC BY-SA 3.0
2015-01-02T17:01:24.090Z
As a side note, you really want to interpolate a frame rather than just using the frame to one side.  2 frames in sequence being the same will be a noticeable stutter.
AJ Henderson
CC BY-SA 3.0
2015-01-02T17:22:53.263Z
Using something like Twixtor would attempt to interpolate movement from the two frames either side - might be a better option than just repeating or interpolating a crossfade of two adjacent frames? http://www.revisionfx.com/products/twixtor/
tomh
CC BY-SA 4.0
2019-09-02T22:49:06.620Z
Highly related: https://video.stackexchange.com/questions/23589/replace-black-frames-with-previous-frames https://video.stackexchange.com/questions/28407/replace-all-completely-black-frames-with-interpolated-frames
BlueRaja - Danny Pflughoeft
<video-editor>
Any nice tool for replacing all black frames in a video with another frame
Naetmul
# Any nice tool for replacing all black frames in a video with another frame

(Asked by: username_0 on 2015-01-01)

My video file has some blackish frames (it's almost black, but not exactly black, i.e. #000000), and I want to filter out them, replacing them with neighboring frames.

However, I can't find a simple way to do that.

Of course, I can extract all the frame images from the video file with FFmpeg:

`ffmpeg -i foo.mp4 -an -framerate 30 -s 1280x720 -f image2 foo-%05d.jpeg`

and then list all the blackish images with some image tools, replace them with neighboring-numbered files, and merge them:

`ffmpeg -i foo.mp4 -vn -acodec copy foo.m4a`  
`ffmpeg -i foo.m4a -f image2 -framerate 30 -i foo-%05d.jpeg -r 30 -s 1280x720 output.mp4`

but this will create over 10,000 intermediate image files and it is also time-consuming.

Is there any software or programming language that helps this work?

I am using Windows x64.

**Tags:** video-editor

> **Comments:**
> **username_1**: Remember that to do the image processing, whatever software you use is going to have to decode each frame so it's always going to be time consuming. Writing them to disk as separate image files will take more time I suppose, but you could perhaps use something like a named pipe to keep it in memory, depending on what image tools you use.  
> **username_2**: As a side note, you really want to interpolate a frame rather than just using the frame to one side.  2 frames in sequence being the same will be a noticeable stutter.  
> **username_3**: Using something like Twixtor would attempt to interpolate movement from the two frames either side - might be a better option than just repeating or interpolating a crossfade of two adjacent frames? http://www.revisionfx.com/products/twixtor/  
> **username_4**: Highly related: https://video.stackexchange.com/questions/23589/replace-black-frames-with-previous-frames https://video.stackexchange.com/questions/28407/replace-all-completely-black-frames-with-interpolated-frames  

---

## 1 answers

### Answer by username_3 on 2015-01-01. Score: 3 | Accepted answer

You could do this in After Effects.  There are lots of examples online where people use Expressions to examine the average brightness of each frame, then act upon it.  For example, you could add a marker to each frame that fell below a certain threshold level. Or make the frame's opacity go to 0%, to allow another track to show through from behind.
Tutorial examples here:
http://markos.co.nz/2012/05/after-effects-expression-use-the-luminance-of-one-layer-to-drive-another-layer/
http://blogs.adobe.com/aftereffects/2009/07/color-sampler-using-sampleimag.html
Here's an example expression I wrote based on the blog posts above -
threshold = 0.01;  // this is the threshold below which the track becomes transparent - black = 0.
driverLayer = thisComp.layer("Pre-comp 1");
samplePoint = [0,0]; // examine from the top left pixel
sampleSize = [640,360]; // examine the full comp size (in this case 640x360)
lightnessSample = driverLayer.sampleImage(samplePoint,sampleSize);  // this samples the image as rgb
driverLightness = rgbToHsl(lightnessSample)[2]; // converts to hue, saturation and lightness, then only takes lightness
if (driverLightness >=threshold) transform.opacity = 100 else 0; // tell opacity to be either 100 or zero depending upon the threshold point chosen in line 1

I applied this to the opacity layer.  You would need to adjust the threshold value, and the size of your comp ( in my case 640x360). (the red solid below was to see whether the opacity change was working.  I guess you would put a copy of your video showing the previous frame here?  Will look as though the video got stuck though...


---

