<p><strong>nadermx/backgroundremover</strong></p>
<p><a href="https://github.com/nadermx/backgroundremover" rel="nofollow noreferrer">https://github.com/nadermx/backgroundremover</a></p>
<p>This is now the top Google hit, I installed version 0.2.9 successfully on Ubuntu 24.10 with:</p>
<pre><code>pipx install backgroundremover
</code></pre>
<p>Unfortunately, there's a bug mentioned at <a href="https://github.com/nadermx/backgroundremover/issues/116" rel="nofollow noreferrer">https://github.com/nadermx/backgroundremover/issues/116</a> which prevents the most common use cases from working (mp4 -&gt; mp4 with background replaced by image or another video), but the author replied there and might fix it.</p>
<p>The following did work <a href="https://github.com/cirosantilli/media/raw/master/Ciro_Santilli_selfie_in_cycling_kit_with_old_cottages.mp4" rel="nofollow noreferrer">with this test video (~1 MB, 5.7 s, 1024x768, 162 frames)</a>:</p>
<pre><code>time backgroundremover -i in.mp4 -tv -o out.mov
</code></pre>
<p>which you can download with:</p>
<pre><code>wget -O in.mp4 https://github.com/cirosantilli/media/raw/master/Ciro_Santilli_selfie_in_cycling_kit_with_old_cottages.mp4
</code></pre>
<p>but it is not very useful (as a final use case) as it produces a huge .mov file (238 MB for an 1 MB input file! 2m25s conversion time on Lenovo ThinkPad P14s amd), and you can't output formats like .mp4 directly like that because .mp4 does not support transparency unlike .mov: <a href="https://github.com/nadermx/backgroundremover/issues/92" rel="nofollow noreferrer">https://github.com/nadermx/backgroundremover/issues/92</a> and I don't know how to use that .mov later on to e.g. replace transparency by a fixed color, though I'm sure there's a way as usual with ffmpeg.</p>
<p>Another thing that worked was their GIF output:</p>
<pre><code>backgroundremover -i in.mp4 -tg o out.gif
</code></pre>
<p>Quality wise, there's are a few glaring artifacts on my simple test case, so it's not perfect. But new better models can always come along later on, currently you can choose between <code>u2net</code>, <code>u2netp</code>, <code>u2net_human_seg</code>. So we understand that this is the base model currently supported: <a href="https://github.com/xuebinqin/U-2-Net" rel="nofollow noreferrer">https://github.com/xuebinqin/U-2-Net</a> , and the exact task that it attempts to solve is &quot;Salient Object Detection&quot;, because e.g. it attempts to select &quot;objects held by people&quot; as part of the foreground just like the person.</p>
<p><a href="https://i.sstatic.net/etQXW.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/etQXW.gif" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/h0pvW.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/h0pvW.gif" alt="enter image description here" /></a></p>
<p>Related:</p>
<ul>
<li><a href="https://askubuntu.com/questions/651309/software-to-automatically-remove-background-from-images">https://askubuntu.com/questions/651309/software-to-automatically-remove-background-from-images</a></li>
<li><a href="https://www.reddit.com/r/linux/comments/oso31y/remove_background_from_video_and_images_with_a/" rel="nofollow noreferrer">https://www.reddit.com/r/linux/comments/oso31y/remove_background_from_video_and_images_with_a/</a></li>
<li><a href="https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python">https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python</a></li>
</ul>
<p>Related: <a href="https://stackoverflow.com/questions/41923492/cant-import-moviepy-editor">https://stackoverflow.com/questions/41923492/cant-import-moviepy-editor</a></p>

CC BY-SA 4.0
2024-05-14T07:43:01.163Z
I find it hard to believe that every video conferencing app I have used (Teams, Meet, etc.) all seem to have this functionality which works very well even with a moving background, and yet this is the best that is available?
Michael
CC BY-SA 4.0
2024-05-14T07:46:40.973Z
@Michael these things are hard to benchmark, I wonder how the proprietary models would perform for my input. My input was also particularly challenging compared to a typical user sitting down facing forward on conference. Do ping me if you find a better one.
Ciro Santilli OurBigBook.com
CC BY-SA 4.0
2024-05-14T08:55:22.467Z
I haven't been able to confirm this, but I suspect that some webcam hardware such as on Macbooks and iPads may have some kind of IR depth sensors, which would greatly reduce the complexity of filtering out the background.  However, I haven't been able to find how one could access this information from something like ffmpeg to do this filtering oneself.
Michael
CC BY-SA 4.0
2024-05-14T09:23:34.040Z
@Michael OK, could be. Some of them have IR for sure, here I show a ffmpeg example: https://askubuntu.com/questions/348838/how-to-check-available-webcams-from-the-command-line/848390#848390
Ciro Santilli OurBigBook.com
CC BY-SA 4.0
2024-05-14T10:46:34.157Z
Thanks!  Sadly, it does not appear that IR/depth information on MacOS (e.g. from TrueDepth cameras) is available using ffmpeg...
Michael
CC BY-SA 4.0
2024-04-20T22:42:50.143Z
2024-12-28T17:03:46.997Z
2024-12-28T17:03:46.997Z
Ciro Santilli OurBigBook.com
<p>There is a reason why monotone background are used, they allow one to determine where the object is, and where is the background. As far as i know no took exists, which can always determine what is the background, and what is the object that is actually being filmed.</p>
<p>If the background is really static (not moving in the frame, rather than simply always being behind the person), then you could attempt to extract frames and determine the pixels that keep their value above some arbitrary percentage of frames. You then can create a mask based on if the pixels of the specific frame are the same as the value of the 'stable pixels', and blur in that mask.</p>
<p>The the background is moving in the frame due to camera movement or changes in some manner, this becomes very difficult, and i may even guess impossible in some situations. Also even if it will work, it will need to be double checked to see that you won't get false positives. You will definitely get false negatives near the person's head.</p>
<p>In the even you have resources for development and really need it to be done, you could go neural network route. Be prepared to provide neural net lots and lots of training data. For that you would need to film akin to the videos that you will be processing, but doing that in front of the green screen. This will allow you to determine where the head truly is in each frame, because once you put in the background it will be the job of the neural net to figure out where the head is, and your job to tell it whether it is correct or not during the training, and to be able to do that in testing to see how well it has learnt to identify the human head.</p>
<p>Good luck.</p>

CC BY-SA 4.0
2022-03-18T11:38:04.813Z
But then, how do Zoom, Teams etc do to separate you from the background in a live call?  I would like to use that same type of software and apply to some training videos we have where the background is distracting
kxtronic
CC BY-SA 4.0
2021-12-11T06:13:14.017Z
2021-12-11T06:13:14.017Z
v010dya
<p>I'm looking for an ffmpeg-like tool to blur/remove/change a static background from a &quot;talking head&quot; video, on the command line with on-device processing.</p>
<p>Essentially,</p>
<pre><code>ffmpeg-like-tool [--blur-background | --remove-background | --change-background] --background image.jpg --in input.mp4 --out output.mp4
</code></pre>
<p>Does such a tool exist yet?</p>

CC BY-SA 4.0
2022-05-03T15:01:27.293Z
Related: https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python
Hermann
<ffmpeg><software-recommendation><greenscreen><command-line>
How to remove background from video without green screen on the command line?
Marek Kowalczyk
# How to remove background from video without green screen on the command line?

(Asked by: username_0 on 2021-03-27)

I'm looking for an ffmpeg-like tool to blur/remove/change a static background from a "talking head" video, on the command line with on-device processing.
Essentially,
ffmpeg-like-tool [--blur-background | --remove-background | --change-background] --background image.jpg --in input.mp4 --out output.mp4

Does such a tool exist yet?

**Tags:** ffmpeg, software-recommendation, greenscreen, command-line

> **Comments:**
> **username_1**: Related: https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python  

---

## 2 answers

### Answer by username_2 on 2024-04-20. Score: 1

nadermx/backgroundremover
https://github.com/nadermx/backgroundremover
This is now the top Google hit, I installed version 0.2.9 successfully on Ubuntu 24.10 with:
pipx install backgroundremover

Unfortunately, there's a bug mentioned at https://github.com/nadermx/backgroundremover/issues/116 which prevents the most common use cases from working (mp4 -> mp4 with background replaced by image or another video), but the author replied there and might fix it.
The following did work with this test video (~1 MB, 5.7 s, 1024x768, 162 frames):
time backgroundremover -i in.mp4 -tv -o out.mov

which you can download with:
wget -O in.mp4 https://github.com/cirosantilli/media/raw/master/Ciro_Santilli_selfie_in_cycling_kit_with_old_cottages.mp4

but it is not very useful (as a final use case) as it produces a huge .mov file (238 MB for an 1 MB input file! 2m25s conversion time on Lenovo ThinkPad P14s amd), and you can't output formats like .mp4 directly like that because .mp4 does not support transparency unlike .mov: https://github.com/nadermx/backgroundremover/issues/92 and I don't know how to use that .mov later on to e.g. replace transparency by a fixed color, though I'm sure there's a way as usual with ffmpeg.
Another thing that worked was their GIF output:
backgroundremover -i in.mp4 -tg o out.gif

Quality wise, there's are a few glaring artifacts on my simple test case, so it's not perfect. But new better models can always come along later on, currently you can choose between u2net, u2netp, u2net_human_seg. So we understand that this is the base model currently supported: https://github.com/xuebinqin/U-2-Net , and the exact task that it attempts to solve is "Salient Object Detection", because e.g. it attempts to select "objects held by people" as part of the foreground just like the person.


Related:

https://askubuntu.com/questions/651309/software-to-automatically-remove-background-from-images
https://www.reddit.com/r/linux/comments/oso31y/remove_background_from_video_and_images_with_a/
https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python

Related: https://stackoverflow.com/questions/41923492/cant-import-moviepy-editor

> **Comments:**
> **username_3**: I find it hard to believe that every video conferencing app I have used (Teams, Meet, etc.) all seem to have this functionality which works very well even with a moving background, and yet this is the best that is available?  
> **username_2**: @username_3 these things are hard to benchmark, I wonder how the proprietary models would perform for my input. My input was also particularly challenging compared to a typical user sitting down facing forward on conference. Do ping me if you find a better one.  
> **username_3**: I haven't been able to confirm this, but I suspect that some webcam hardware such as on Macbooks and iPads may have some kind of IR depth sensors, which would greatly reduce the complexity of filtering out the background.  However, I haven't been able to find how one could access this information from something like ffmpeg to do this filtering oneself.  
> **username_2**: @username_3 OK, could be. Some of them have IR for sure, here I show a ffmpeg example: https://askubuntu.com/questions/348838/how-to-check-available-webcams-from-the-command-line/848390#848390  
> **username_3**: Thanks!  Sadly, it does not appear that IR/depth information on MacOS (e.g. from TrueDepth cameras) is available using ffmpeg...  

---

### Answer by username_4 on 2021-12-11. Score: 0

There is a reason why monotone background are used, they allow one to determine where the object is, and where is the background. As far as i know no took exists, which can always determine what is the background, and what is the object that is actually being filmed.

If the background is really static (not moving in the frame, rather than simply always being behind the person), then you could attempt to extract frames and determine the pixels that keep their value above some arbitrary percentage of frames. You then can create a mask based on if the pixels of the specific frame are the same as the value of the 'stable pixels', and blur in that mask.

The the background is moving in the frame due to camera movement or changes in some manner, this becomes very difficult, and i may even guess impossible in some situations. Also even if it will work, it will need to be double checked to see that you won't get false positives. You will definitely get false negatives near the person's head.

In the even you have resources for development and really need it to be done, you could go neural network route. Be prepared to provide neural net lots and lots of training data. For that you would need to film akin to the videos that you will be processing, but doing that in front of the green screen. This will allow you to determine where the head truly is in each frame, because once you put in the background it will be the job of the neural net to figure out where the head is, and your job to tell it whether it is correct or not during the training, and to be able to do that in testing to see how well it has learnt to identify the human head.

Good luck.

> **Comments:**
> **username_5**: But then, how do Zoom, Teams etc do to separate you from the background in a live call?  I would like to use that same type of software and apply to some training videos we have where the background is distracting  

---

