<p>Since you've not accepted the other two answers (which I thought were very good) because they focus on motion blur and parallax/ perspective, I'm guessing that the "wobble" you're talking about due to the rolling shutter effect that CMOS sensors introduce to footage.  </p>

<p>In a nutshell, CMOS sensors scan an each frame of an exposure very quickly from top to bottom.  While the entire exposure occurs within the 1/60th of a second it takes to capture a 30fps 180° shutter image, for example, the capture is not instantaneous.  This causes the image to skew temporally, an effect which is most visibly noticeable when the camera pans quickly or shakes.</p>

<p>The best way to avoid this problem is to use a camera with "global shutter".  CCD sensors in old camcorders did not suffer from rolling shutter, but because CMOS outpaced the development of CCD and because not many people noticed rolling shutter, global shutter cameras are now a niche market.  </p>

<p>Congratulations.  You fall squarely within this niche.  But consider yourself in good company.  One of the primary markets for global shutter cameras is the VFX industry.  This is because rolling shutter is A) particularly problematic in matchmoving software like Boujou and B) particularly noticeable after software stabilization.</p>

<p>The trouble with rolling shutter, is that it doesn't actually cause a perspective shift, although it may appear to.  For example, point your cell phone camera out the driver's side window of a moving car and snap a picture (while someone else is driving, obviously).  The image will slant in one direction.  Now snap a picture out the passenger window.  The image will slant in the opposite direction.  So while you could easily correct this "slant" in Photoshop with edit->transform->skew or edit-transform->perspective, there's no way for software to perform the correction automatically, because it doesn't know which window you were shooting from.
<img src="https://i.sstatic.net/tTOQb.jpg" alt="car window cmos"></p>

<p>Another example: Take a picture of a spinning propeller, and the blades will appear to curve, not slant.  Rolling shutter distorts images in ways that post software can't compensate because the variables include camera motion, scene motion, and direction of scan.
<img src="https://i.sstatic.net/UDPpW.jpg" alt="spinning prop cmos"></p>

<p>I realize this information doesn't help fix footage you've already shot, but hopefully it will help point you in the right direction.  I'm not too familiar with many software packages to fix this problem.  After Effects has a "rolling shutter" plugin which may be sufficient for your needs.  Perhaps someone can recommend some others.</p>

CC BY-SA 3.0
2015-01-06T08:35:45.000Z
I usually accept answers when I try things suggested and it's work. It can happen after a long time (i.e. the next time I need to deshake video). And the answers look edited recently, I still have to re-read them.
Vi.
CC BY-SA 3.0
2015-01-06T08:37:22.563Z
Maybe it's that `parallax/perspective`. The last time I was reading the answers it was not mentioned (or I was reading too carelessly).
Vi.
CC BY-SA 3.0
2015-01-06T08:38:54.430Z
Rolling shutter? Does it mean the distorsion in vertical movements and horisontal movements will be different?
Vi.
CC BY-SA 3.0
2015-01-06T14:40:45.373Z
@Vi - In fairness, I did kind of stealth add the parallax/perspective bit to mine in response to your comments.  Perspective shifting/parallax is what you are talking about from the images you added (which are super helpful by the way.)
AJ Henderson
CC BY-SA 3.0
2015-01-06T14:56:17.377Z
I expect this perspective thing should not be too tricky to add to simplish stabilizer. Maybe I'll request the author of `vid.stab` to implement it (or even try myself). I'll revisit the question later...
Vi.
CC BY-SA 3.0
2015-01-28T00:43:05.430Z
There is a perspective shift. If you look at the water tank you'll see that it is more skewed than the poles in the background, because it moves more in the frame during the exposure. Applying a skew to the whole image would over-correct the background. To correct it completely you'd have to skew the foreground and background separately. This would require adding in the missing background. Motion analysis could tell you which parts of the frame are moving and correct them according to their speed, and content-sensitive fill could plug the gaps. So it's possible, but difficult.
stib
CC BY-SA 3.0
2015-01-06T08:23:39.447Z
2015-01-06T16:16:25.570Z
2015-01-06T16:16:25.570Z
Jason Conrad
<p>The problem is that this is what the original footage looked like.  While you can stabilize the objects themselves by tracking, you still have motion blur from the movement because the objects moved within the time each frame was being captured.  Additionally, you also have perspective changes if the camera is moving in 3d space in addition to having the angle change.</p>

<p>This motion blur appears regardless of the stabilization as the stabilization simply repositioned the video to keep objects in the same place, but you can't get rid of the motion blur without completely synthesizing the image to produce something that resembles what the frame would have looked like without any motion blur.  This is both exceedingly difficult and also extremely likely to introduce as many artifacts as it fixes.</p>

<p>Similarly, the perspective issues result in a shifting perspective without corresponding movement as the camera position changes are counteracted by providing a perspective correction, but the contents of the video are still viewing from the wrong angle, so what portion of the surface is visible changes very disconcertingly.</p>

<p>This is most obvious if we take a hypothetical situation where we have a camera looking at a dice.  We start looking straight down at two dots.  Then, after a few seconds, the camera suddenly gets jerked around to the front and we are looking at 4 dots.  To smooth that camera motion, we would need to make it look like the camera movement was smooth between the two positions, however, we don't actually know the 3d geometry of the dice, we just know that it shifted relative to other things in the scene.  We can't simply apply some distortion to each frame, because half of our frames only show the 2 dots and half our frames only show the 4 dots.  </p>

<p>Most image stabilization is limited to working with whatever detail is available in frame and warping and adjusting it (and in advanced cases, independently warping individual objects on screen).  We can shift the perspective of the image so it seems kind of like the camera is moving, but we'll still see the two dots until suddenly it changes to 4 dots.  The camera movement will feel a bit more smooth, but the perspective was telling us we were looking at half way between the side and the top while the number of dots shows us to be looking at the top.  This disjoint can't be fixed without synthesizing new data, which is super hard.</p>

<p>The best bet to avoid motion blur is to use the fastest possible shutter speed when capturing footage.  This will reduce the amount of motion blur present and help make stabilized footage look much better.  After the fact, it is exceedingly far more difficult to correct for.</p>

<p>For perspective, there isn't much you can do other than trying to limit the amount of changes in position of the camera.</p>

<p>There are some interesting bits of research by organizations like <a href="http://research.microsoft.com/en-us/um/redmond/projects/hyperlapse/" rel="nofollow">Microsoft Research</a> (there is also an interesting tidbit they have <a href="http://research.microsoft.com/en-us/um/redmond/projects/hyperlapse/igcomp/" rel="nofollow">here</a> about what the currently available state of the art is) can build point clouds that can be used to correct for perspective issues if the video is complete enough, but such research is exceedingly ground breaking and mind numbingly complex, so you won't find it in the open source world yet (or even the commercial one quite yet.)</p>

CC BY-SA 3.0
2015-01-04T19:15:07.450Z
The question is not about motion blur. Looks like `vitstabtransform` corrects: 1. position; 2. rotation; but it is not engouh. Other geometrical distorsion creeps in. I want to know how is it called and how do I get rid of it.
Vi.
CC BY-SA 3.0
2015-01-04T19:30:18.463Z
OK. That is a crappy stabilization algorithm plus motion blur plus perspective issues. It is doing a perspective transform but the image is still shot from off angle so the visual information is inconsistent with the angle it appears to be from.
AJ Henderson
CC BY-SA 3.0
2015-01-04T19:50:35.573Z
So what should I look at when searching for non-crappy open source stabilization?
Vi.
CC BY-SA 3.0
2015-01-04T22:33:05.933Z
Stabilisation is such a complex problem, mathematically and programmatically, to put it bluntly: there ain't any open source project with enough research budget (or money full stop) to make anything close to the commercial products (referring to After Effects' Warp Stabiliser, which still would struggle with this shot - and it's got all of Adobe R& D behind it).
nchpmn
CC BY-SA 3.0
2015-01-06T02:12:50.083Z
There are more professional programs that have largely solved the stabilization problem, Smoke and Nuke being two examples, but you're looking at thousands of dollars for that software unless you're a student.
KC McLaughlin
CC BY-SA 3.0
2015-01-06T06:05:27.050Z
@KCMcLaughlin - have they solved the perspective shift that Vi was talking about?  That MS research stuff is pretty recent and it was majorly ground breaking as nobody else has been even close to perspective correction as it involves having to completely rebuild the 3d scene and motion estimate from a single 2d viewpoint.  I would have expected to have heard about it if anyone had fully solved a perfect way to recreate and motion estimate it.
AJ Henderson
CC BY-SA 3.0
2015-01-06T14:08:53.133Z
@AJHenderson I'm not aware that those companies have made use of that recent research, my comment was more in response to nchpmn who was using AE as his best example for stabilization when there are several much better programs.
KC McLaughlin
CC BY-SA 3.0
2015-01-06T14:34:44.580Z
@KCMcLaughlin - ah yeah, there are definitely better options than warp stabilizer.  There is a reason that Mocha was (is it still?) included with AE.
AJ Henderson
CC BY-SA 3.0
2015-01-04T18:37:48.003Z
2015-01-06T14:49:09.587Z
2015-01-06T14:49:09.587Z
AJ Henderson
<p>When a camera is recording video, it takes many still photographs per second. </p>

<p>For each of these photographs, the camera opens its shutter (either mechanical or electronic), and allows light to hit the sensor for an amount of time. During this time, any object that moves, or any motion in the camera, is captured as motion blur. Imagine watching an athlete running - theirs legs, arms are blurred in the image. </p>

<p>Stabilising footage uses algorithms to detect and calculate the movement in the shot. It then moves and warps the image so the greatest amount possible doesn't move (I.e. The background - a "steady" shot). However, the motion blur captured in the image still exists - this creates the jitters and wobbles you are seeing. </p>

<p>Unfortunately, it's also (to my professional knowledge) impossible to remove - it's part of the original, recorded image. </p>

<p>If you have manual control over the camera, you can try changing the shutter speed to a faster value (<em>1/big number</em> or <em>small number°</em>) to reduce the amount of motion blur recorded. However, this can introduce other problems, including strobing, from a severe lack of motion blur, and making it more difficult to get an exposure in dark scenes. </p>

CC BY-SA 3.0
2015-01-04T14:54:30.323Z
It is not about motion blur. It is about remaining geometric deformations that are not rotation and moving. Motion blue is also a problem, but the separate one.
Vi.
CC BY-SA 3.0
2015-01-04T08:41:35.137Z
2015-01-04T08:41:35.137Z
nchpmn
<p>The common name for the effect you're talking about is "Jello" and it's caused by rolling shutter in combination with camera shake.</p>

<p>Each frame is distorted by the rolling shutter because, for example, the left edge of your fence is at a different place left to right as the "shutter" selects the pixels as it rolls from top to bottom of the frame, resulting in a trapeziodal distortion of that frame.  Next frame the shake may cause it to distort in the other direction.  Over the course of several frames this results in the "Jello" effect with vertical lines apparently wobbling side to side.  Running stabilising filters sometimes makes it look even worse because the wobble can be hidden a bit when there's a lot of other global motion going on.</p>

<p>Here's an OSS solution, it's a plugin for VirtualDub, it corrects shake, roll, and rolling shutter / Jello:</p>

<p><a href="http://www.guthspot.se/video/deshaker.htm" rel="nofollow noreferrer">http://www.guthspot.se/video/deshaker.htm</a></p>

CC BY-SA 4.0
2017-10-17T06:23:36.563Z
2018-07-25T00:41:22.273Z
2018-07-25T00:41:22.273Z
Wil
<p>When I filter a shaky video through deshaking filter (that fixes frames positions and rotations), the resulting video looks "wobbly" (<a href="http://defun.co/media/2014-10-16/CzsLN2GPRd.gif" rel="noreferrer">example gif</a>), i.e. while there are no per-frame displacements or tilt, there is still per-frame deformation.</p>

<p>How is it called and how do I get rid of it?</p>

<p><strong>Note: The question is not about motion blur, it is about geometrical deformation that is not rotation and moves.</strong></p>

<p><strong>Update:</strong> I have experimented a bit with various videos and found out the case where the problem is visible the most.</p>

<p>If camera is held at right angle to the object (<img src="https://i.sstatic.net/PNF0T.png" alt="camera at right angle">), I get only motion blur after deshaking.</p>

<p>But of camera is held at angle to the surface (<img src="https://i.sstatic.net/p7E1z.png" alt="camera at angle">), then I got this type of distorsion when moving the camera: <img src="https://i.sstatic.net/RhxiH.png" alt="distorsion">.</p>

CC BY-SA 3.0
2015-01-04T09:23:15.613Z
Try [Mercalli](http://vegasaur.com/Mercalli) from ProDAD. They claim that it is able to remove wobble and jello.
altarvic
CC BY-SA 3.0
2015-01-04T14:52:44.223Z
@altarvic, I looking mostly for CLI-based FFmpeg-facing things like `vidstabdetect/vidstabtransform`. `a stand-alone Windows application that works in Windows Vista, Windows 7 or Windows 8` is not nice too. Maybe I'll try this in Wine...
Vi.
<stabilisation>
Why deshaken video looks "wobbly" and how to fix it
Vi.
# Why deshaken video looks "wobbly" and how to fix it

(Asked by: username_0 on 2015-01-04)

When I filter a shaky video through deshaking filter (that fixes frames positions and rotations), the resulting video looks "wobbly" ([example gif](http://defun.co/media/2014-10-16/CzsLN2GPRd.gif)), i.e. while there are no per-frame displacements or tilt, there is still per-frame deformation.

How is it called and how do I get rid of it?

**Note: The question is not about motion blur, it is about geometrical deformation that is not rotation and moves.**

**Update:** I have experimented a bit with various videos and found out the case where the problem is visible the most.

If camera is held at right angle to the object (![camera at right angle](https://i.sstatic.net/PNF0T.png)), I get only motion blur after deshaking.

But of camera is held at angle to the surface (![camera at angle](https://i.sstatic.net/p7E1z.png)), then I got this type of distorsion when moving the camera: ![distorsion](https://i.sstatic.net/RhxiH.png).

**Tags:** stabilisation

> **Comments:**
> **username_1**: Try [Mercalli](http://vegasaur.com/Mercalli) from ProDAD. They claim that it is able to remove wobble and jello.  
> **username_0**: @username_1, I looking mostly for CLI-based FFmpeg-facing things like `vidstabdetect/vidstabtransform`. `a stand-alone Windows application that works in Windows Vista, Windows 7 or Windows 8` is not nice too. Maybe I'll try this in Wine...  

---

## 4 answers

### Answer by username_2 on 2015-01-06. Score: 4

Since you've not accepted the other two answers (which I thought were very good) because they focus on motion blur and parallax/ perspective, I'm guessing that the "wobble" you're talking about due to the rolling shutter effect that CMOS sensors introduce to footage.

In a nutshell, CMOS sensors scan an each frame of an exposure very quickly from top to bottom. While the entire exposure occurs within the 1/60th of a second it takes to capture a 30fps 180° shutter image, for example, the capture is not instantaneous. This causes the image to skew temporally, an effect which is most visibly noticeable when the camera pans quickly or shakes.

The best way to avoid this problem is to use a camera with "global shutter". CCD sensors in old camcorders did not suffer from rolling shutter, but because CMOS outpaced the development of CCD and because not many people noticed rolling shutter, global shutter cameras are now a niche market.

Congratulations. You fall squarely within this niche. But consider yourself in good company. One of the primary markets for global shutter cameras is the VFX industry. This is because rolling shutter is A) particularly problematic in matchmoving software like Boujou and B) particularly noticeable after software stabilization.

The trouble with rolling shutter, is that it doesn't actually cause a perspective shift, although it may appear to. For example, point your cell phone camera out the driver's side window of a moving car and snap a picture (while someone else is driving, obviously). The image will slant in one direction. Now snap a picture out the passenger window. The image will slant in the opposite direction. So while you could easily correct this "slant" in Photoshop with edit->transform->skew or edit-transform->perspective, there's no way for software to perform the correction automatically, because it doesn't know which window you were shooting from.
![car window cmos](https://i.sstatic.net/tTOQb.jpg)

Another example: Take a picture of a spinning propeller, and the blades will appear to curve, not slant. Rolling shutter distorts images in ways that post software can't compensate because the variables include camera motion, scene motion, and direction of scan.
![spinning prop cmos](https://i.sstatic.net/UDPpW.jpg)

I realize this information doesn't help fix footage you've already shot, but hopefully it will help point you in the right direction. I'm not too familiar with many software packages to fix this problem. After Effects has a "rolling shutter" plugin which may be sufficient for your needs. Perhaps someone can recommend some others.

> **Comments:**
> **username_0**: I usually accept answers when I try things suggested and it's work. It can happen after a long time (i.e. the next time I need to deshake video). And the answers look edited recently, I still have to re-read them.  
> **username_0**: Maybe it's that `parallax/perspective`. The last time I was reading the answers it was not mentioned (or I was reading too carelessly).  
> **username_0**: Rolling shutter? Does it mean the distorsion in vertical movements and horisontal movements will be different?  
> **username_3**: @Vi - In fairness, I did kind of stealth add the parallax/perspective bit to mine in response to your comments.  Perspective shifting/parallax is what you are talking about from the images you added (which are super helpful by the way.)  
> **username_0**: I expect this perspective thing should not be too tricky to add to simplish stabilizer. Maybe I'll request the author of `vid.stab` to implement it (or even try myself). I'll revisit the question later...  
> **username_4**: There is a perspective shift. If you look at the water tank you'll see that it is more skewed than the poles in the background, because it moves more in the frame during the exposure. Applying a skew to the whole image would over-correct the background. To correct it completely you'd have to skew the foreground and background separately. This would require adding in the missing background. Motion analysis could tell you which parts of the frame are moving and correct them according to their speed, and content-sensitive fill could plug the gaps. So it's possible, but difficult.  

---

### Answer by username_3 on 2015-01-04. Score: 4

The problem is that this is what the original footage looked like. While you can stabilize the objects themselves by tracking, you still have motion blur from the movement because the objects moved within the time each frame was being captured. Additionally, you also have perspective changes if the camera is moving in 3d space in addition to having the angle change.

This motion blur appears regardless of the stabilization as the stabilization simply repositioned the video to keep objects in the same place, but you can't get rid of the motion blur without completely synthesizing the image to produce something that resembles what the frame would have looked like without any motion blur. This is both exceedingly difficult and also extremely likely to introduce as many artifacts as it fixes.

Similarly, the perspective issues result in a shifting perspective without corresponding movement as the camera position changes are counteracted by providing a perspective correction, but the contents of the video are still viewing from the wrong angle, so what portion of the surface is visible changes very disconcertingly.

This is most obvious if we take a hypothetical situation where we have a camera looking at a dice. We start looking straight down at two dots. Then, after a few seconds, the camera suddenly gets jerked around to the front and we are looking at 4 dots. To smooth that camera motion, we would need to make it look like the camera movement was smooth between the two positions, however, we don't actually know the 3d geometry of the dice, we just know that it shifted relative to other things in the scene. We can't simply apply some distortion to each frame, because half of our frames only show the 2 dots and half our frames only show the 4 dots.

Most image stabilization is limited to working with whatever detail is available in frame and warping and adjusting it (and in advanced cases, independently warping individual objects on screen). We can shift the perspective of the image so it seems kind of like the camera is moving, but we'll still see the two dots until suddenly it changes to 4 dots. The camera movement will feel a bit more smooth, but the perspective was telling us we were looking at half way between the side and the top while the number of dots shows us to be looking at the top. This disjoint can't be fixed without synthesizing new data, which is super hard.

The best bet to avoid motion blur is to use the fastest possible shutter speed when capturing footage. This will reduce the amount of motion blur present and help make stabilized footage look much better. After the fact, it is exceedingly far more difficult to correct for.

For perspective, there isn't much you can do other than trying to limit the amount of changes in position of the camera.

There are some interesting bits of research by organizations like [Microsoft Research](http://research.microsoft.com/en-us/um/redmond/projects/hyperlapse/) (there is also an interesting tidbit they have [here](http://research.microsoft.com/en-us/um/redmond/projects/hyperlapse/igcomp/) about what the currently available state of the art is) can build point clouds that can be used to correct for perspective issues if the video is complete enough, but such research is exceedingly ground breaking and mind numbingly complex, so you won't find it in the open source world yet (or even the commercial one quite yet.)

> **Comments:**
> **username_0**: The question is not about motion blur. Looks like `vitstabtransform` corrects: 1. position; 2. rotation; but it is not engouh. Other geometrical distorsion creeps in. I want to know how is it called and how do I get rid of it.  
> **username_3**: OK. That is a crappy stabilization algorithm plus motion blur plus perspective issues. It is doing a perspective transform but the image is still shot from off angle so the visual information is inconsistent with the angle it appears to be from.  
> **username_0**: So what should I look at when searching for non-crappy open source stabilization?  
> **username_5**: Stabilisation is such a complex problem, mathematically and programmatically, to put it bluntly: there ain't any open source project with enough research budget (or money full stop) to make anything close to the commercial products (referring to After Effects' Warp Stabiliser, which still would struggle with this shot - and it's got all of Adobe R& D behind it).  
> **username_6**: There are more professional programs that have largely solved the stabilization problem, Smoke and Nuke being two examples, but you're looking at thousands of dollars for that software unless you're a student.  
> **username_3**: @KCMcLaughlin - have they solved the perspective shift that Vi was talking about?  That MS research stuff is pretty recent and it was majorly ground breaking as nobody else has been even close to perspective correction as it involves having to completely rebuild the 3d scene and motion estimate from a single 2d viewpoint.  I would have expected to have heard about it if anyone had fully solved a perfect way to recreate and motion estimate it.  
> **username_6**: @AJHenderson I'm not aware that those companies have made use of that recent research, my comment was more in response to username_5 who was using AE as his best example for stabilization when there are several much better programs.  
> **username_3**: @KCMcLaughlin - ah yeah, there are definitely better options than warp stabilizer.  There is a reason that Mocha was (is it still?) included with AE.  

---

### Answer by username_5 on 2015-01-04. Score: 1

When a camera is recording video, it takes many still photographs per second.

For each of these photographs, the camera opens its shutter (either mechanical or electronic), and allows light to hit the sensor for an amount of time. During this time, any object that moves, or any motion in the camera, is captured as motion blur. Imagine watching an athlete running - theirs legs, arms are blurred in the image.

Stabilising footage uses algorithms to detect and calculate the movement in the shot. It then moves and warps the image so the greatest amount possible doesn't move (I.e. The background - a "steady" shot). However, the motion blur captured in the image still exists - this creates the jitters and wobbles you are seeing.

Unfortunately, it's also (to my professional knowledge) impossible to remove - it's part of the original, recorded image.

If you have manual control over the camera, you can try changing the shutter speed to a faster value (*1/big number* or *small number°*) to reduce the amount of motion blur recorded. However, this can introduce other problems, including strobing, from a severe lack of motion blur, and making it more difficult to get an exposure in dark scenes.

> **Comments:**
> **username_0**: It is not about motion blur. It is about remaining geometric deformations that are not rotation and moving. Motion blue is also a problem, but the separate one.  

---

### Answer by username_7 on 2017-10-17. Score: 1

The common name for the effect you're talking about is "Jello" and it's caused by rolling shutter in combination with camera shake.

Each frame is distorted by the rolling shutter because, for example, the left edge of your fence is at a different place left to right as the "shutter" selects the pixels as it rolls from top to bottom of the frame, resulting in a trapeziodal distortion of that frame. Next frame the shake may cause it to distort in the other direction. Over the course of several frames this results in the "Jello" effect with vertical lines apparently wobbling side to side. Running stabilising filters sometimes makes it look even worse because the wobble can be hidden a bit when there's a lot of other global motion going on.

Here's an OSS solution, it's a plugin for VirtualDub, it corrects shake, roll, and rolling shutter / Jello:

<http://www.guthspot.se/video/deshaker.htm>


---

