<p>... Depends...</p>
<p>There are many concepts that need to be differentiated. Let me explore a bit.</p>
<p><strong>Information</strong></p>
<p>The first one is that we need to think about an image as information. So we need to ask first. Is the information lost? Is the information modified? Is it improved? Keep this concept in mind.</p>
<p><strong>Scaling</strong></p>
<p>And now let me apply this idea to another word. Scaling.</p>
<p>Scaling &quot;per se&quot; does not modify the initial information. It only adapts it to conform to another viewing size or display.</p>
<p>Take your 320x240 video file and do nothing with it. Only play it on any monitor to full screen. The video is exactly the same. If viewed on a large screen you will surely notice the pixels, but that is your perception. The information on the video file is exactly the same.</p>
<hr />
<p>Now let us actually scale the image so it is no longer a 320x240 file. Normally you would like to use round multiples. 2x, 3x, 4x. If you use a &quot;nearest neighbor&quot; scaling method, the information will be <strong>exactly the same</strong>.</p>
<p>A 240 P image 4 times would be a 960P image. But scaling it to 1080 has a compromise of how you scale some pixels, the factor is 4.5x so, either some pixels are scaled 4 times and some other 5 times, or you average some values.</p>
<p><strong>Resampling</strong></p>
<p>And here is the key. An operation you are now performing is not scaling, it is resampling. Taking some values, and converting them into other values.</p>
<p>If I say that John has $20 and Mike has $10. I have the exact information.</p>
<p>If I say that the average they have is $15 it destroys the original information. It could be that one has $30 and the other has nothing. Or that the amounts are inverted.</p>
<p>So, resampling is the one that compromises the information.</p>
<hr />
<blockquote>
<p>mainly from stretching pixels.</p>
</blockquote>
<p>Here you are probably referring to the aspect ratio. Which is another thing to consider. It can be a resampling maintaining the aspect ratio. Keep that in mind.</p>
<blockquote>
<p>I know anecdotally that if you take a 320x240 video file and upscale it to 1920x1080 that the quality is lost</p>
</blockquote>
<p>Not really.</p>
<p>First of all, you <em>perceive</em> it as <em>low quality</em> because you are now used to see HD images, so you are adding how you perceive the resulting image.</p>
<p>Again, see that image on a watch, or use it on a nanotechnology screen on a label on a soda can, and you would find it impressive.</p>
<p>Quality is a process, not an end result... necessarily</p>
<hr />
<p><strong>The algorithm</strong></p>
<p>Our capacity to percive information depends on the information itself.</p>
<p>Our eyes can detect detail if there is contrast. And we percive this contrast on the edges of a shape.</p>
<p>Some resampling algorithms try to enhance the detail on the borders of the shapes. Some try to enhance it, some try to generate new information guessing what values should be there.</p>
<hr />
<p>At the end there is always a compromise on what do you expect to see, what tools are at our disposal, and how much of the original information you can, or want to keep.</p>
<hr />
<p>P.S. Downsampling always reduces the ammount of information, but it is only important if the information is relevant.</p>
<p>A 4k video file of a completly white wall will probably not have relevant information, so we can resample safely to SD. But if we have a small text painted there, probably this text will be lost on the downsampling.</p>

CC BY-SA 4.0
2021-03-28T15:24:17.267Z
2021-03-28T15:24:17.267Z
Rafael
<p>May i understand your question as</p>
<blockquote>
<p>How do i have someone/something that understands the missing puzzles from
my upscaled images?</p>
</blockquote>
<p>If so, you are talking some intelligence/experience in upscaling or AI-upscaling. Then, plz take a look of <a href="https://sites.google.com/view/aiupscalingtutorials/ai-upscaling" rel="nofollow noreferrer">AI Upscalers</a>; it mentions</p>
<ul>
<li>An open source AI upscaler (super sampler) based on a generative adversarial network architecture.</li>
<li>An open source image enhancer that seeks to restore lost texture details from known types</li>
<li>Commercial super sampler that uses artificial intelligence</li>
</ul>
<p>Good luck!</p>

CC BY-SA 4.0
2020-10-28T00:50:11.887Z
2020-10-28T00:50:11.887Z
xer-rex
<p>I was wondering if there was any hard number evidence for upscaling or downscaling footage to a certain raster size, and what happens to the quality or if any degradation or improvements are made to the output.</p>
<p>I know anecdotally that if you take a <code>320x240</code> video file and upscale it to <code>1920x1080</code> that the quality is lost - mainly from stretching pixels.</p>
<p>I know anecdotally that if you take a <code>3840x2160</code> video file and downscale it to <code>1024x576</code> that the quality in the larger areas remain, and but the finer details can be lost when compressed into a smaller pixel area.</p>
<p>I'm currently in an environment workplace that uses only SD outputs, but for some reason there is a push to take the incoming media (which can range from <code>320x240</code> all the way to 8k) and conform it to <code>1920x1080</code>. When it is imported into the editor, it is an SD project, and burnt to DVD-Video.</p>
<p>For some reason the explanation of matching our output raster or device (DVD-Video) doesn't seem as sufficient evidence, so I wanted to know if there were any numbers or studies or factual papers I could use!</p>
<hr />
<p>For the most part assume the conversions are performed with &quot;off the shelf&quot; converters, which most likely run <code>ffmpeg</code> in their processing. All resizes with in the applications too would be default - so <em>Bicubic interpolation</em> is most common.</p>

<encoding><aspect><decoding><standards-converter>
What actually happens to footage that is upscaled or downscaled?
markb
# What actually happens to footage that is upscaled or downscaled?

(Asked by: username_0 on 2020-10-27)

I was wondering if there was any hard number evidence for upscaling or downscaling footage to a certain raster size, and what happens to the quality or if any degradation or improvements are made to the output.

I know anecdotally that if you take a `320x240` video file and upscale it to `1920x1080` that the quality is lost - mainly from stretching pixels.

I know anecdotally that if you take a `3840x2160` video file and downscale it to `1024x576` that the quality in the larger areas remain, and but the finer details can be lost when compressed into a smaller pixel area.

I'm currently in an environment workplace that uses only SD outputs, but for some reason there is a push to take the incoming media (which can range from `320x240` all the way to 8k) and conform it to `1920x1080`. When it is imported into the editor, it is an SD project, and burnt to DVD-Video.

For some reason the explanation of matching our output raster or device (DVD-Video) doesn't seem as sufficient evidence, so I wanted to know if there were any numbers or studies or factual papers I could use!

---

For the most part assume the conversions are performed with "off the shelf" converters, which most likely run `ffmpeg` in their processing. All resizes with in the applications too would be default - so *Bicubic interpolation* is most common.

**Tags:** encoding, aspect, decoding, standards-converter


---

## 2 answers

### Answer by username_1 on 2021-03-28. Score: 1

... Depends...

There are many concepts that need to be differentiated. Let me explore a bit.

**Information**

The first one is that we need to think about an image as information. So we need to ask first. Is the information lost? Is the information modified? Is it improved? Keep this concept in mind.

**Scaling**

And now let me apply this idea to another word. Scaling.

Scaling "per se" does not modify the initial information. It only adapts it to conform to another viewing size or display.

Take your 320x240 video file and do nothing with it. Only play it on any monitor to full screen. The video is exactly the same. If viewed on a large screen you will surely notice the pixels, but that is your perception. The information on the video file is exactly the same.

---

Now let us actually scale the image so it is no longer a 320x240 file. Normally you would like to use round multiples. 2x, 3x, 4x. If you use a "nearest neighbor" scaling method, the information will be **exactly the same**.

A 240 P image 4 times would be a 960P image. But scaling it to 1080 has a compromise of how you scale some pixels, the factor is 4.5x so, either some pixels are scaled 4 times and some other 5 times, or you average some values.

**Resampling**

And here is the key. An operation you are now performing is not scaling, it is resampling. Taking some values, and converting them into other values.

If I say that John has $20 and Mike has $10. I have the exact information.

If I say that the average they have is $15 it destroys the original information. It could be that one has $30 and the other has nothing. Or that the amounts are inverted.

So, resampling is the one that compromises the information.

---

> mainly from stretching pixels.

Here you are probably referring to the aspect ratio. Which is another thing to consider. It can be a resampling maintaining the aspect ratio. Keep that in mind.

> I know anecdotally that if you take a 320x240 video file and upscale it to 1920x1080 that the quality is lost

Not really.

First of all, you *perceive* it as *low quality* because you are now used to see HD images, so you are adding how you perceive the resulting image.

Again, see that image on a watch, or use it on a nanotechnology screen on a label on a soda can, and you would find it impressive.

Quality is a process, not an end result... necessarily

---

**The algorithm**

Our capacity to percive information depends on the information itself.

Our eyes can detect detail if there is contrast. And we percive this contrast on the edges of a shape.

Some resampling algorithms try to enhance the detail on the borders of the shapes. Some try to enhance it, some try to generate new information guessing what values should be there.

---

At the end there is always a compromise on what do you expect to see, what tools are at our disposal, and how much of the original information you can, or want to keep.

---

P.S. Downsampling always reduces the ammount of information, but it is only important if the information is relevant.

A 4k video file of a completly white wall will probably not have relevant information, so we can resample safely to SD. But if we have a small text painted there, probably this text will be lost on the downsampling.


---

### Answer by username_2 on 2020-10-28. Score: 0

May i understand your question as

> How do i have someone/something that understands the missing puzzles from
> my upscaled images?

If so, you are talking some intelligence/experience in upscaling or AI-upscaling. Then, plz take a look of [AI Upscalers](https://sites.google.com/view/aiupscalingtutorials/ai-upscaling); it mentions

* An open source AI upscaler (super sampler) based on a generative adversarial network architecture.
* An open source image enhancer that seeks to restore lost texture details from known types
* Commercial super sampler that uses artificial intelligence

Good luck!


---

